My first „Hello World!” processor!

This guide will show you how to create the new processor of yours, add some layers to it and set their properties.

A bit of disclaimer: this guide will continuously evolve as developer documentations do. But hey, it exists! So it worth to come back from time to time or subscribe to its change-notification. The latest state of this text always reflects the HEAD version of branch „szotsaki”. Generally speaking, if something doesn't work for you, just merge and recompile everything.

So, the basics

You want to write a processor which does some calculations. You also want to show your work and later maybe interact with it on the GUI. For all of these you are needed to deal with three classes:

  1. Processor: of course this is the most important one. All the essence of your work goes mostly here.
  2. Layer: if you want some show-off, you'll also need this. Or these. You have the possibility to attach more layers to a processor. Or none. It's truly your choice whether you want to show something to the world.
  3. ProcessorSlot: last but not least, a structure is needed which encompasses these aforementioned classes together. You won't need to care about this much though, its defaults are just perfect.

Processors

First, start with the Processor. It has a name and arbitrary number of custom properties. The processor-related files are stored with the model, in the MarbeCommon/Processors directory. To create your first processor, let's call it FirstProcessor, you have to subclass from the abstract Processor class like this:

class WIN_DLL_DECLSPEC FirstProcessor : public Processor
{
    Q_OBJECT
public:
    FirstProcessor(QObject *parent = nullptr);
    ~FirstProcessor() = default;
}

The Q_OBJECT macro is necessary because of some boring administration reasons. The parent pointer is necessary because Qt maintains a hierarchy of the declared objects for easier deletion among others.

The WIN_DLL_DECLSPEC is necessary because Windows is able to load library classes only when either the __declspec(dllexport) or __declspec(dllimport) is present before the class name. This macro helps decide when to use which variation. Since it is needed only in classes exported to DLLs, places other than MarbleCommon it is not needed to be defined. For further information see this MSDN article. NB. this is a Microsoft-specific and Windows-only workaround.

Here is, how to define this class:

FirstProcessor::FirstProcessor(QObject *parent)
    : Processor("My first Processor", parent)
{
}

We pass the name of our processor (which will be shown on the GUI) to the parent constructor.

Layers

Now, head towards to the Layers.

The resemblance is uncanny with the processors. You will just have to create a new Layer class of yours (call it, for the sake of simplicity, FirstLayer), make it a descendant of the Layer abstract class and pass your layer's name to the parent constructor. The layer-related files lie with other GUI files, in the GrainAutLine/Layers directory. A very basic implementation would look like this:

class FirstLayer : public Layer
{
    Q_OBJECT
public:
    explicit FirstLayer(QObject *parent = nullptr);
    ~FirstLayer() = default;
}
FirstLayer::FirstLayer(QObject *parent)
    : Layer("My first layer", parent)
{
}

The ProcessorSlot

Let's finish with the encapsulating class, the ProcessorSlot.

First, you need to shortly name your work you did so far. The name of the compilation will appear only in code but it should be expressive for other developers. Register this name in the ProcessorSlot::Type enumeration. Now you can create a new specific instance of ProcessorSlot: please, usher yourself to the hall of the ProcessorSlot constructor and supplement the switch-case structure with the name of your choice.

The Processor_ variable will hold your custom processor. Create a new instance of it:

Processor_ = std::make_shared<FirstProcessor>();

The Layers vector holds all the layers you want to use later. Previously we created only one, FirstLayer, but feel free to either add it multiple times or create many different layers later and add them here:

Layers.push_back(std::make_shared<FirstLayer>());

Note that layers are optional extensions for a Processor determined to show the internal state of the latter. Therefore it's not necessary to attach a layer at all.

The almighty encapsulator: the SlotManager

Maybe there's a slight chance you've been wondering if there is an encapsulation class for Layers and Processors (this is the ProcessorSlot), is there a similar container for many ProcessorSlots? Yes, there is. It is called SlotManager which, not surprisingly, manages the life cycles of ProcessorSlots.

When you are ready with your modifications so far, you are needed to register your ProcessorSlot in SlotManager. This is quite easy, just add your class to the following container in its constructor like this:

NonElementarySlots.push_back(
    std::make_shared<ProcessorSlot>(ProcessorSlot::Type::MyFirstProcessorSlot, this)
  );

Make sure that you replace MyFirstProcessorSlot with the name you used previously in the enumeration.

Finishing

You created a new Processor and a new Layer. Then you added them to the their encapsulation class, ProcessorSlot. After that you registered this processor slot at their manager, SlotManager.

Now, we are ready. Oh, just one thing: make sure that your processor computes and layer draws. How easy to say that, isn't it? Well, it will surely be a joyful journey. For the beginning, define and override the following functions:

virtual bool FirstProcessor::Step(ProcessingStateDescriptor& psd) override;
virtual void FirstLayer::Render(const ProcessingStateDescriptor& psd) override;

Have a nice time coding!

Reference documentation

From now on, we are going through the different trifles of the API to make you get most out of the programming interface.

ProcessorSlot

As we saw previously, a ProcessorSlot encompasses exactly one Processor and an arbitrary number of Layers.

Elementary ProcessorSlots

An elementary ProcessorSlot has the following properties:

To make a ProcessorSlot elementary, just set the Elementary_ variable to true in the ProcessorSlot constructor. Just like the following example does:

[…]
case Type::ShowImage:
        Processor_ = std::make_shared<ShowImageProcessor>(this);
        Layers.push_back(std::make_shared<ShowImageLayer>(this));
        Elementary_ = true;
        break;
[…]

This property cannot be modified at run-time.

Please, pay attention to instantiate your elementary ProcessorSlot in the right section of the SlotManager constructor. A comment line indicates where these are going.

Processor

Processor properties

A processor is a rather complex entity, therefore it needs some properties to fine-tune itself. These properties are shown on the GUI and freely configurable by the user. All properties have a unique name, a default value and of course they hold their current value.

To add a property to your Processor, simply call the AddProperty method in your Processor constructor.

Currently, there are two different kinds of properties you can use:

  1. Float type: You can store a float value inside with a minimum and maximum range provided.

    void AddProperty(const QString& name,
                 const float minimum,
                 const float maximum,
                 const float default_);

    Reading its value is possible by its name with the following function:

    float GetFloatPropertyValue(const QString& name);

  2. Boolean type: You can store a bool value inside.

    void AddProperty(const QString& name,
                 const bool default_);

    Reading its value is possible by its name with the following function:

    bool GetBoolPropertyValue(const QString& name);

Overriding Run()

A Processor works this simplified way: when a mouse click occurred or a new image file was loaded, the Run method starts and calls Step until it returns false. If you need more elaborate control, you can override this method. Its default implementation looks like the following:

void Processor::Run(ProcessingStateDescriptor& psd)
{
    while(Step(psd) == true)
        ;

    emit ComputationReady(psd);
}

Please, always emit the ComputationReady signal when the Run process finishes.

Mouse handling

Subscribing to the mouse events

There are four types of mouse events exist:

Constant Description
QEvent::MouseButtonPress Mouse press
QEvent::MouseButtonRelease Mouse release
QEvent::MouseButtonDblClick Mouse press again
QEvent::MouseMove Mouse move

The MouseButtonRelease is e.g. useful for one-shot actions like running the processor after a click was made at a certain coordinate. While the MouseMove is best for continuous actions like drawing. You can freely subscribe to each of them independently with your processor.

By default, a Processor doesn't receive these mouse events. To do so, add the desired ones to the SubscribedMouseEvents unordered_set structure like this:

SubscribedMouseEvents = {QEvent::MouseButtonRelease, QEvent::MouseMove};

You may add them directly at the construction time of your class, although you can modify these values any time you want; the changes will come into effect before the next run of your Processor.

Please note that regardless whether you are subscribed to the mouse events, your non-elementary Processor won't receive them if it's not the currently selected one.

Reacting to a mouse event

Supposing you're already subscribed to the desired mouse events, your processor will receive them in the following virtual function:

virtual void MouseEventOccurred(const QMouseEvent& event,
                                const QString& canvasName,
                                ProcessingStateDescriptor& psd)

The first variable contains the event itself. You can determine its type by using the type() function of its. For example if you are subscribed to the previously mentioned events, you can distinguish between them in the following way:

void MyFirstProcessor::MouseEventOccurred(const QMouseEvent &event, const QString &canvasName, ProcessingStateDescriptor &psd)
{
    // Refresh the MouseEvent variable both on QEvent::MouseButtonRelease and QEvent::MouseMove
    MouseEvent = event;

    if (event.type() == QEvent::MouseButtonRelease) {
        MouseClicked(canvasName, psd);
    }
}

The default implementation of this function stores the QMouseEvent object into the MouseEvent member variable. In case you don't need extra handling of the events, you may not want to override this function, just read the MouseEvent member from your Processor.

By default a non-elementary Processor won't run after a mouse or keyboard event occurred. To force it, simply call RunInThread() where you want to start the computing like in this custom mouse click handler:

void MyFirstProcessor::MouseClicked(const QString &canvasName, ProcessingStateDescriptor &psd)
{
    Q_UNUSED(canvasName)

    // This processor is also used to trigger re-rendering without modifying something
    RunInThread(psd);
}

Warning! Always make sure that the ComputationReady signal is emitted from a non-elementary processor's keyboard or mouse event handling function. It can be either a RunInThread call which automatically emits this signal at the end of its run or emit it manually. This guarantees the follow-up elementary processors will run and the screen is refreshed. An example for this:

void MyFirstProcessor::MouseEventOccurred(const QMouseEvent &event, const QString &canvasName, ProcessingStateDescriptor &psd)
{
    if (event.type() == QEvent::MouseButtonRelease) {
        RunInThread(psd);
    } else {
        emit ComputationReady(psd);
    }
}

Getting the current mouse coordinates

When a mouse event occurred and the processor is set to receive these events, by default the MouseEvent member contains the current event with its position. Use the pos or localPos functions to read out these coordinates. Please, refer to the QMouseEvent and QPoint documentation on handling these values.

Please note that because of performance reasons functions screenPos, windowPos and globalPos always return with QPoint(0, 0).

The top-left image coordinate is (0, 0) and the given coordinates are always resize-agnostics.

Layers

A Layer renders a bit of portion of the main image which will be blended into the renderings of other layers. After all the layers created their own cv::Mat image, the so called ImageProvider collects and flattens them into the final image which will be shown to user on the canvas(es).

Accessing to its Processor

It is an easy task since it involves only calling the GetProcessor getter from anywhere inside the Layer's function. Calling this getter it's your responsibility to specify exactly that type of Processor which belongs to the current Layer. The return value of this function is a std::shared_ptr so handle it like a normal pointer type.

An example which calls the custom CreateMagic() function of your FirstProcessor:

GetProcessor<FirstProcessor>()->CreateMagic();

Sequences

Each Layer has a number, called Sequence. This defines the order of layers to render on each other. You can query this number if you need by using member function GetSequence but please, by any means, do not ever change this value!

Rendering

This is the function in you are free to leverage your creativity. One thing you need to pay a bit attention to: emit the iAmReadyWithRendering signal, here called RenderReady if you want your layer image to be shown. For example like this:

void FirstLayer::Render(const ProcessingStateDescriptor& psd)
{
    // Generate image to cv::Mat
    const cv::Mat image = psd.ExportToImage("");

    emit RenderReady(GetTargetCanvas(), GetSequence(), image, GetOpacity(), CanvasCacheBase::allDirty);
}

The signal signature is the following:

void RenderReady(const QString& canvasName, // Name of the canvas to draw on
                 const int layerSequence,   // Sequence ID of the current layer
                 const cv::Mat& mat,        // The finished image in cv::Mat format
                 const float opacity,       // Opacity of the current layer set on the GUI
                 const cv::Rect dirtyRoI);  // Dirty region of the layer (which was modified)

Caching

Since converting many cv::Mats from the processors on every pixel the mouse advanced to can be a really time consuming operation, always use cache. If the Layer knows that the image didn't change, recall the previously rendered cv::Mat from member variable Cache. An example showing this:

void ShowImageLayer::Render(const ProcessingStateDescriptor &psd)
{
    cv::Rect dirtyRoI = CanvasCacheBase::allDirty;

    if (Cache.data == nullptr) {
        cv::Mat imgOriginal = cv::imread(psd.ImageFileName, CV_LOAD_IMAGE_COLOR);
        Q_ASSERT(imgOriginal.data);

        Cache = imgOriginal;
    } else {
        dirtyRoI = CanvasCacheBase::noneDirty;
    }

    emit RenderReady(GetTargetCanvas(), GetSequence(), Cache, GetOpacity(), dirtyRoI);
}

Beware, that your condition on invalidating the cache might be different.

Supposedly, your cache depends on the very image file. If a new one got loaded, the cache has to be invalidated. In this case you simply have to override the appropriate function FileOpened. It is defined in classes Processor, Layer and ImageProvider:

virtual void Processor::FileOpened(const QString& fileName);
virtual void Layer::FileOpened(const QString& fileName);
void ImageProvider::FileOpened(const QString& fileName);

As soon as this function is invoked, you can be sure a new image file has been loaded already. Be careful as this function gets called both when an image file or a .prc file was opened. If it's needed for your class to differentiate between these formats, use the provided fileName parameter.

The dirty region attribute is a standard cv::Rect. If you just want to indicate that all of the pixels of your layer were modified, use CanvasCacheBase::allDirty. On the other hand, to indicate that none of the pixels were modified, use CanvasCacheBase::noneDirty as the example above.

Additional canvases

The system is capable of rendering layers not only on the main canvas but on arbitrary number of canvases.

Note that if your processor is set to receive mouse events, these are forwarded from all the canvases. The originating canvas name is provided in the emitted signal. Be sure to distinguish among them while processing mouse events if your ProcessorSlot has layers which render on different canvases.

Default canvases

There are two pre-defined canvases are already in the program, namely:

  1. The main canvas: this is that the users first meet after an image file is opened. By default all the layers render onto this canvas. If you specifically want to declare your layer to render on this one, use its unique name mainCanvas.
  2. The second, by default supported one is the supplementary canvas. This is a little canvas mainly used as a magnifier of the original picture on the right hand side of the application. If you want your layer to render on this one, use its name supplementaryCanvas.

Making layers to render on a specific canvas

A layer is capable of rendering only on one canvas which has to be defined at compile time and must not be changed during application run.

By default it renders on the main canvas. If you want to specify another render target, override the following virtual function of your layer:

virtual QString GetTargetCanvas() const;

Then return the target canvas name, e.g. supplementaryCanvas:

QString MyFirstLayer::GetTargetCanvas() const
{
    return QStringLiteral("supplementaryCanvas");
}

Creating a new canvas

Only two simple places you have to extend when you want to create a new canvas:

  1. QML: First of all, decide where the new canvas has to be displayed and then create a PsdCanvas element there in the appropriate QML file. Set the object name to your new canvas name. Something like this:

    PsdCanvas {
    objectName: "histogram"
    Layout.preferredWidth: 250
    Layout.preferredHeight: Layout.preferredWidth / 2
}

  2. Right after that, just create a new entry in the ImageProvider::CanvasNames static container with your new canvas name. It is located in the beginning of the ImageProvider.cpp source file.

After these two steps, you can use your layers to draw on your custom canvas described in the sub-chapter above.

Debugging

For every sane developer the time will come to log. Everything. Ok, maybe not the register contents.

GrainAutLine supports logging in two simultaneous ways:

Each log message has to fall into one of these four distinct types:

Constant Description
QtDebugMsg A message generated by the qDebug() function.
QtInfoMsg A message generated by the qInfo() function.
QtWarningMsg A message generated by the qWarning() function.
QtCriticalMsg A message generated by the qCritical() function.
QtFatalMsg A message generated by the qFatal() function.
After printing this out, the application terminates.

For using the Qt debug facility, please refer to the links above.

A category can also be associated with a message. If you want to distinguish messages based on categories, you just need to use the functions above with C like, qCDebug(), qCWarning() and so forth while providing the category as a parameter. An example:

qCDebug(funCategory) << "Take one down and pass it around," << remainingBeers << "bottles of beer on the wall.";

Don't forget that qDebug() takes care of the spaces and newlines for you.

Defining own categories

Creating and maintaining your own category has it perks. For example you can create debug messages everywhere that only you can see and don't disturb the others with them.

To create a new category, just append the following line to LogCategories.h:

Q_DECLARE_LOGGING_CATEGORY(developerName)

And its pair to LogCategories.cpp:

Q_LOGGING_CATEGORY(developerName, "dev.developerName", QtWarningMsg)

From now on you are eligible to use logging functions like this: qCDebug(developerName) or qCCritical(developerName).

  1. The first parameter (developerName) is the short name of the category which you can use in the program.
  2. The second parameter (dev.developerName) will be shown on the GUI and on the console run-time. Since it's a regular string, it can contain spaces.
  3. The third parameter tells the minimum logging level. The example above logs only warning messages and everything above it (i.e. ignoring QtDebugMsg-type calls).

Unfortunately, turning them on and off conveniently from a preferences file is currently not supported. Please, check back later for it.

GPU information

If you would like to access some of the information the target GPU provides, use the static methods of GpuChecker class:

The HAVE_CUDA macro is defined if CUDA support is available at compile time.

The following globally defined macros help track down some various GPU-related function call errors: