Overview

In this challenging project, given a diagram, the goal is to extract semantic relationships that are within it. Any technique, include machine learning, can be used.

Scope of this project is limited to block diagrams and flowcharts that include geometric shapes, arrows and connectors. Diagrams may be in colour or monochrome. Elements within the diagram may be plain or may have 3D shadows or other styling.

Semantic relationships are typically one of the following:

• Block A is connected to Block B with bidirectional arrow.
• Block A and Block B are hierarchically within Block C.
• Block D is 25% bigger than Block E.
• Block H is a circle.
• Block F contains the text "A/D Converter".
• Block A is an input block.
• Block C connects to Block E based on condition "a = 2".
• The forward flow is from Block A to Block K.
• The arrow from Block J to Block B goes against the flow: it may represent feedback or a closed loop.

The output from the algorithm should be complete. In other words, it should be possible to recreate the complete image from the semantic description, although the final image may look quite different from the original. Recreating the image is not part of this project but it may be necessary to do this for testing purpose.

Deliverables

Project can be implemented in any language. Code should adopt a modular design. Provide basic documentation and example outputs based on a diverse set of inputs.

Code should support the following:

• A module to pre-process the image. This may adjust contrast, sharpen the text, convert to monochrome, etc.
• A module that implements the main algorithm to extract semantic relationships. If you're adopting a ML approach, separate the training from the prediction workflows.
• Define the format in which semantic relationships will be saved to file.
• A module to export and import semantic relationships in the format defined above.