Custom Neural Networks

If you plan to use your own neural network with KerasAnalyser, consider the internal structure of the analysis pipeline.

flowchart LR KerasAnalyser -- ImageStack --> QThreadpool; QThreadpool --> QRunnable1; QThreadpool --> QRunnable2; QThreadpool --> QRunnable3; QRunnable1 -- Computation --> Interpreter; QRunnable2 -- Computation --> Interpreter; QRunnable3 -- Computation --> Interpreter;

The ImageAnalyser is responsible for collecting the necessary amount of images and only starting analysis if all images have been received. KerasAnalyser will notify you if you load a model that does not fit the settings in the acquisition. These image stacks are forwarded to QRunnables in a QThreadpool that can concurrently analyse the images and send the results to the interpreter.

Computation

Depening on your neural network you might need a special QRunnable that allows for some functionality around the model. This can be subclassed from KerasWorker. See examples in examples.analysers like KerasTilingWorker and KerasRescaleWorker. The dataflow is visualized here:

flowchart TB ImageStack --> prep[/prepare_images/] -- "data['pixels']" --> Inference data("data: {'pixels': network_input, 'e.g. tile_positions': xxx}") event>Signal] Inference --> extr[/extract_decision_parameter/] --> ev1>"new_decision_parameter.emit()"] --EventBus --> Interpreter Inference --> post[/post_process_output/] --> ev2>"new_network_image.emit()"] --> GUI Inference(Model predict_on_batch) prep -- data --> post

If you want your worker to be available in the AnalyserGUI, you can add its location to the settings.json file.

"eda_plugin": {
    "analysers": {
        "keras": {
            "worker_modules": [
                "eda_plugin.examples.analysers",
                "your_worker.module_here"
            ],
}}}

Resource allocation

Resource allocation is handeled by a QThreadpool, using its tryStart() method, this ensures that images are skipped if no resources are available for analysis. The number of availble threads can be overwritten by subclassing the KerasAnalyser and setting self.threadpool.setMaxThreadCount(number_of_threads) in the __init__ function after initializing super().__init__().