My first „Hello World!” processor!
This guide will show you how to create the new processor of yours, add some layers to it and set their properties.
A bit of disclaimer: this guide will continuously evolve as developer documentations do. But hey, it exists! So it worth to come back from time to time or subscribe to its change-notification. The latest state of this text always reflects the HEAD version of branch „szotsaki”. Generally speaking, if something doesn't work for you, just merge and recompile everything.
So, the basics
You want to write a processor which does some calculations. You also want to show your work and later maybe interact with it on the GUI. For all of these you are needed to deal with three classes:
-
Processor
: of course this is the most important one. All the essence of your work goes mostly here. -
Layer
: if you want some show-off, you'll also need this. Or these. You have the possibility to attach more layers to a processor. Or none. It's truly your choice whether you want to show something to the world. -
ProcessorSlot
: last but not least, a structure is needed which encompasses these aforementioned classes together. You won't need to care about this much though, its defaults are just perfect.
Processors
First, start with the Processor
. It has a name and arbitrary number of custom properties. The processor-related files are stored with the model, in the MarbeCommon/Processors directory. To create your first processor, let's call it FirstProcessor
, you have to subclass from the abstract Processor
class like this:
class WIN_DLL_DECLSPEC FirstProcessor : public Processor
{
Q_OBJECT
public:
FirstProcessor(QObject *parent = nullptr);
~FirstProcessor() = default;
}
The Q_OBJECT
macro is necessary because of some boring administration reasons. The parent
pointer is necessary because Qt maintains a hierarchy of the declared objects for easier deletion among others.
The WIN_DLL_DECLSPEC
is necessary because Windows is able to load library classes only when either the __declspec(dllexport)
or __declspec(dllimport)
is present before the class name. This macro helps decide when to use which variation. Since it is needed only in classes exported to DLLs, places other than MarbleCommon
it is not needed to be defined. For further information see this MSDN article. NB. this is a Microsoft-specific and Windows-only workaround.
Here is, how to define this class:
FirstProcessor::FirstProcessor(QObject *parent)
: Processor("My first Processor", parent)
{
}
We pass the name of our processor (which will be shown on the GUI) to the parent constructor.
Layers
Now, head towards to the Layers
.
The resemblance is uncanny with the processors. You will just have to create a new Layer
class of yours (call it, for the sake of simplicity, FirstLayer
), make it a descendant of the Layer
abstract class and pass your layer's name to the parent constructor. The layer-related files lie with other GUI files, in the GrainAutLine/Layers directory. A very basic implementation would look like this:
class FirstLayer : public Layer
{
Q_OBJECT
public:
explicit FirstLayer(QObject *parent = nullptr);
~FirstLayer() = default;
}
FirstLayer::FirstLayer(QObject *parent)
: Layer("My first layer", parent)
{
}
The ProcessorSlot
Let's finish with the encapsulating class, the ProcessorSlot
.
First, you need to shortly name your work you did so far. The name of the compilation will appear only in code but it should be expressive for other developers. Register this name in the ProcessorSlot::Type
enumeration. Now you can create a new specific instance of ProcessorSlot
: please, usher yourself to the hall of the ProcessorSlot
constructor and supplement the switch-case structure with the name of your choice.
The Processor_
variable will hold your custom processor. Create a new instance of it:
Processor_ = std::make_shared<FirstProcessor>();
The Layers
vector holds all the layers you want to use later. Previously we created only one, FirstLayer
, but feel free to either add it multiple times or create many different layers later and add them here:
Layers.push_back(std::make_shared<FirstLayer>());
Note that layers are optional extensions for a Processor
determined to show the internal state of the latter. Therefore it's not necessary to attach a layer at all.
The almighty encapsulator: the SlotManager
Maybe there's a slight chance you've been wondering if there is an encapsulation class for Layers
and Processors
(this is the ProcessorSlot
), is there a similar container for many ProcessorSlots
? Yes, there is. It is called SlotManager
which, not surprisingly, manages the life cycles of ProcessorSlots
.
When you are ready with your modifications so far, you are needed to register your ProcessorSlot
in SlotManager
. This is quite easy, just add your class to the following container in its constructor like this:
NonElementarySlots.push_back(
std::make_shared<ProcessorSlot>(ProcessorSlot::Type::MyFirstProcessorSlot, this)
);
Make sure that you replace MyFirstProcessorSlot
with the name you used previously in the enumeration.
Finishing
You created a new Processor
and a new Layer
. Then you added them to the their encapsulation class, ProcessorSlot
. After that you registered this processor slot at their manager, SlotManager
.
Now, we are ready. Oh, just one thing: make sure that your processor computes and layer draws. How easy to say that, isn't it? Well, it will surely be a joyful journey. For the beginning, define and override the following functions:
- for the computation itself:
virtual bool FirstProcessor::Step(ProcessingStateDescriptor& psd) override;
- and for the drawing:
virtual void FirstLayer::Render(const ProcessingStateDescriptor& psd) override;
Have a nice time coding!
Reference documentation
From now on, we are going through the different trifles of the API to make you get most out of the programming interface.
ProcessorSlot
As we saw previously, a ProcessorSlot
encompasses exactly one Processor
and an arbitrary number of Layers
.
Elementary ProcessorSlots
An elementary ProcessorSlot
has the following properties:
- Its
Processor
always runs regardless of everything. This is necessary for making such fundamental tasks done like the raw image rendering. This means that:- its
Processor
will run after a non-elementaryProcessor
; - and unconditionally after a key or mouse event occurred.
- its
- Its
Processor
name doesn't show up in the drop-down processor selector list on the GUI, so it cannot be selected to run alone. - This infers that an elementary
Processor
cannot have properties configured from the GUI, since its property window will never be displayed. - Nevertheless, it still can have as many layers as you want and these layers are always shown until the user decides to turn them off.
To make a ProcessorSlot
elementary, just set the Elementary_
variable to true
in the ProcessorSlot
constructor. Just like the following example does:
[…]
case Type::ShowImage:
Processor_ = std::make_shared<ShowImageProcessor>(this);
Layers.push_back(std::make_shared<ShowImageLayer>(this));
Elementary_ = true;
break;
[…]
This property cannot be modified at run-time.
Please, pay attention to instantiate your elementary ProcessorSlot
in the right section of the SlotManager
constructor. A comment line indicates where these are going.
Processor
Processor properties
A processor is a rather complex entity, therefore it needs some properties to fine-tune itself. These properties are shown on the GUI and freely configurable by the user. All properties have a unique name, a default value and of course they hold their current value.
To add a property to your Processor
, simply call the AddProperty
method in your Processor
constructor.
Currently, there are two different kinds of properties you can use:
-
Float type: You can store a
float
value inside with a minimum and maximum range provided.void AddProperty(const QString& name, const float minimum, const float maximum, const float default_);
Reading its value is possible by its name with the following function:
float GetFloatPropertyValue(const QString& name);
-
Boolean type: You can store a
bool
value inside.void AddProperty(const QString& name, const bool default_);
Reading its value is possible by its name with the following function:
bool GetBoolPropertyValue(const QString& name);
Overriding Run()
A Processor
works this simplified way: when a mouse click occurred or a new image file was loaded, the Run
method starts and calls Step
until it returns false
. If you need more elaborate control, you can override this method. Its default implementation looks like the following:
void Processor::Run(ProcessingStateDescriptor& psd)
{
while(Step(psd) == true)
;
emit ComputationReady(psd);
}
Please, always emit the ComputationReady
signal when the Run
process finishes.
Mouse handling
Subscribing to the mouse events
There are four types of mouse events exist:
Constant | Description |
---|---|
QEvent::MouseButtonPress | Mouse press |
QEvent::MouseButtonRelease | Mouse release |
QEvent::MouseButtonDblClick | Mouse press again |
QEvent::MouseMove | Mouse move |
The MouseButtonRelease
is e.g. useful for one-shot actions like running the processor after a click was made at a certain coordinate. While the MouseMove
is best for continuous actions like drawing. You can freely subscribe to each of them independently with your processor.
By default, a Processor
doesn't receive these mouse events. To do so, add the desired ones to the SubscribedMouseEvents
unordered_set
structure like this:
SubscribedMouseEvents = {QEvent::MouseButtonRelease, QEvent::MouseMove};
You may add them directly at the construction time of your class, although you can modify these values any time you want; the changes will come into effect before the next run of your Processor
.
Please note that regardless whether you are subscribed to the mouse events, your non-elementary Processor
won't receive them if it's not the currently selected one.
Reacting to a mouse event
Supposing you're already subscribed to the desired mouse events, your processor will receive them in the following virtual
function:
virtual void MouseEventOccurred(const QMouseEvent& event,
const QString& canvasName,
ProcessingStateDescriptor& psd)
The first variable contains the event itself. You can determine its type by using the type()
function of its. For example if you are subscribed to the previously mentioned events, you can distinguish between them in the following way:
void MyFirstProcessor::MouseEventOccurred(const QMouseEvent &event, const QString &canvasName, ProcessingStateDescriptor &psd)
{
// Refresh the MouseEvent variable both on QEvent::MouseButtonRelease and QEvent::MouseMove
MouseEvent = event;
if (event.type() == QEvent::MouseButtonRelease) {
MouseClicked(canvasName, psd);
}
}
The default implementation of this function stores the QMouseEvent
object into the MouseEvent
member variable. In case you don't need extra handling of the events, you may not want to override this function, just read the MouseEvent
member from your Processor
.
By default a non-elementary Processor
won't run after a mouse or keyboard event occurred. To force it, simply call RunInThread()
where you want to start the computing like in this custom mouse click handler:
void MyFirstProcessor::MouseClicked(const QString &canvasName, ProcessingStateDescriptor &psd)
{
Q_UNUSED(canvasName)
// This processor is also used to trigger re-rendering without modifying something
RunInThread(psd);
}
Warning! Always make sure that the ComputationReady
signal is emitted from a non-elementary processor's keyboard or mouse event handling function. It can be either a RunInThread
call which automatically emits this signal at the end of its run or emit it manually. This guarantees the follow-up elementary processors will run and the screen is refreshed. An example for this:
void MyFirstProcessor::MouseEventOccurred(const QMouseEvent &event, const QString &canvasName, ProcessingStateDescriptor &psd)
{
if (event.type() == QEvent::MouseButtonRelease) {
RunInThread(psd);
} else {
emit ComputationReady(psd);
}
}
Getting the current mouse coordinates
When a mouse event occurred and the processor is set to receive these events, by default the MouseEvent
member contains the current event with its position. Use the pos
or localPos
functions to read out these coordinates. Please, refer to the QMouseEvent and QPoint documentation on handling these values.
Please note that because of performance reasons functions screenPos
, windowPos
and globalPos
always return with QPoint(0, 0)
.
The top-left image coordinate is (0, 0) and the given coordinates are always resize-agnostics.
Layers
A Layer
renders a bit of portion of the main image which will be blended into the renderings of other layers. After all the layers created their own cv::Mat
image, the so called ImageProvider
collects and flattens them into the final image which will be shown to user on the canvas(es).
Accessing to its Processor
It is an easy task since it involves only calling the GetProcessor
getter from anywhere inside the Layer's function. Calling this getter it's your responsibility to specify exactly that type of Processor
which belongs to the current Layer
. The return value of this function is a std::shared_ptr
so handle it like a normal pointer type.
An example which calls the custom CreateMagic()
function of your FirstProcessor
:
GetProcessor<FirstProcessor>()->CreateMagic();
Sequences
Each Layer
has a number, called Sequence
. This defines the order of layers to render on each other. You can query this number if you need by using member function GetSequence
but please, by any means, do not ever change this value!
Rendering
This is the function in you are free to leverage your creativity. One thing you need to pay a bit attention to: emit the iAmReadyWithRendering signal, here called RenderReady
if you want your layer image to be shown. For example like this:
void FirstLayer::Render(const ProcessingStateDescriptor& psd)
{
// Generate image to cv::Mat
const cv::Mat image = psd.ExportToImage("");
emit RenderReady(GetTargetCanvas(), GetSequence(), image, GetOpacity(), CanvasCacheBase::allDirty);
}
The signal signature is the following:
void RenderReady(const QString& canvasName, // Name of the canvas to draw on
const int layerSequence, // Sequence ID of the current layer
const cv::Mat& mat, // The finished image in cv::Mat format
const float opacity, // Opacity of the current layer set on the GUI
const cv::Rect dirtyRoI); // Dirty region of the layer (which was modified)
Caching
Since converting many cv::Mats from the processors on every pixel the mouse advanced to can be a really time consuming operation, always use cache. If the Layer
knows that the image didn't change, recall the previously rendered cv::Mat
from member variable Cache
. An example showing this:
void ShowImageLayer::Render(const ProcessingStateDescriptor &psd)
{
cv::Rect dirtyRoI = CanvasCacheBase::allDirty;
if (Cache.data == nullptr) {
cv::Mat imgOriginal = cv::imread(psd.ImageFileName, CV_LOAD_IMAGE_COLOR);
Q_ASSERT(imgOriginal.data);
Cache = imgOriginal;
} else {
dirtyRoI = CanvasCacheBase::noneDirty;
}
emit RenderReady(GetTargetCanvas(), GetSequence(), Cache, GetOpacity(), dirtyRoI);
}
Beware, that your condition on invalidating the cache might be different.
Supposedly, your cache depends on the very image file. If a new one got loaded, the cache has to be invalidated. In this case you simply have to override the appropriate function FileOpened
. It is defined in classes Processor
, Layer
and ImageProvider
:
virtual void Processor::FileOpened(const QString& fileName);
virtual void Layer::FileOpened(const QString& fileName);
void ImageProvider::FileOpened(const QString& fileName);
As soon as this function is invoked, you can be sure a new image file has been loaded already. Be careful as this function gets called both when an image file or a .prc
file was opened. If it's needed for your class to differentiate between these formats, use the provided fileName
parameter.
The dirty region attribute is a standard cv::Rect
. If you just want to indicate that all of the pixels of your layer were modified, use CanvasCacheBase::allDirty
. On the other hand, to indicate that none of the pixels were modified, use CanvasCacheBase::noneDirty
as the example above.
Additional canvases
The system is capable of rendering layers not only on the main canvas but on arbitrary number of canvases.
Note that if your processor is set to receive mouse events, these are forwarded from all the canvases. The originating canvas name is provided in the emitted signal. Be sure to distinguish among them while processing mouse events if your ProcessorSlot has layers which render on different canvases.
Default canvases
There are two pre-defined canvases are already in the program, namely:
- The main canvas: this is that the users first meet after an image file is opened. By default all the layers render onto this canvas. If you specifically want to declare your layer to render on this one, use its unique name
mainCanvas
. - The second, by default supported one is the supplementary canvas. This is a little canvas mainly used as a magnifier of the original picture on the right hand side of the application. If you want your layer to render on this one, use its name
supplementaryCanvas
.
Making layers to render on a specific canvas
A layer is capable of rendering only on one canvas which has to be defined at compile time and must not be changed during application run.
By default it renders on the main canvas. If you want to specify another render target, override the following virtual function of your layer:
virtual QString GetTargetCanvas() const;
Then return the target canvas name, e.g. supplementaryCanvas
:
QString MyFirstLayer::GetTargetCanvas() const
{
return QStringLiteral("supplementaryCanvas");
}
Creating a new canvas
Only two simple places you have to extend when you want to create a new canvas:
-
QML: First of all, decide where the new canvas has to be displayed and then create a
PsdCanvas
element there in the appropriate QML file. Set the object name to your new canvas name. Something like this:PsdCanvas { objectName: "histogram" Layout.preferredWidth: 250 Layout.preferredHeight: Layout.preferredWidth / 2 }
Right after that, just create a new entry in the
ImageProvider::CanvasNames
static container with your new canvas name. It is located in the beginning of theImageProvider.cpp
source file.
After these two steps, you can use your layers to draw on your custom canvas described in the sub-chapter above.
Debugging
For every sane developer the time will come to log. Everything. Ok, maybe not the register contents.
GrainAutLine supports logging in two simultaneous ways:
- both to the standard output (to the
stderr
, to be precise) - and to the GUI.
Each log message has to fall into one of these four distinct types:
Constant | Description |
---|---|
QtDebugMsg | A message generated by the qDebug() function. |
QtInfoMsg | A message generated by the qInfo() function. |
QtWarningMsg | A message generated by the qWarning() function. |
QtCriticalMsg | A message generated by the qCritical() function. |
QtFatalMsg | A message generated by the qFatal() function. After printing this out, the application terminates. |
For using the Qt debug facility, please refer to the links above.
A category can also be associated with a message. If you want to distinguish messages based on categories, you just need to use the functions above with C
like, qCDebug()
, qCWarning()
and so forth while providing the category as a parameter. An example:
qCDebug(funCategory) << "Take one down and pass it around," << remainingBeers << "bottles of beer on the wall.";
Don't forget that qDebug()
takes care of the spaces and newlines for you.
Defining own categories
Creating and maintaining your own category has it perks. For example you can create debug messages everywhere that only you can see and don't disturb the others with them.
To create a new category, just append the following line to LogCategories.h
:
Q_DECLARE_LOGGING_CATEGORY(developerName)
And its pair to LogCategories.cpp
:
Q_LOGGING_CATEGORY(developerName, "dev.developerName", QtWarningMsg)
From now on you are eligible to use logging functions like this: qCDebug(developerName)
or qCCritical(developerName)
.
- The first parameter (
developerName
) is the short name of the category which you can use in the program. - The second parameter (
dev.developerName
) will be shown on the GUI and on the console run-time. Since it's a regular string, it can contain spaces. - The third parameter tells the minimum logging level. The example above logs only warning messages and everything above it (i.e. ignoring
QtDebugMsg
-type calls).
Unfortunately, turning them on and off conveniently from a preferences file is currently not supported. Please, check back later for it.
GPU information
If you would like to access some of the information the target GPU provides, use the static methods of GpuChecker
class:
-
static bool SystemCudaCapable()
: tells whether GrainAutLine has CUDA support compiled in and the current system can run CUDA applications. -
static QList<QString> GetCudaDeviceNames()
: returns a list of CUDA capable devices. -
static bool SetCurrentDeviceId(const int value)
: sets the current CUDA device tovalue
. -
static int GetCurrentDeviceId()
: returns the ID of above. -
static QString GetCurrentDeviceName()
: returns the currently selected CUDA-capable GPU name.
The HAVE_CUDA
macro is defined if CUDA support is available at compile time.
The following globally defined macros help track down some various GPU-related function call errors:
-
cudaCheckError(cudaError_t error)
: prints out the CUDA error message then exits if there is one. This macro is only available when CUDA support is enabled at compile-time. -
nppCheckError(NppStatus result)
: prints the NVIDIA Performance Primitives (NPP) call error message then exits if there is one. Also prints its warnings, but in this case doesn't quit. This macro is only available when CUDA support is enabled at compile-time. -
glCheckError()
: prints if there were OpenGL errors previously. Use this function sparingly because it causes synchronisation. This macro is always available at compile-time, but call only when an OpenGL context is valid.