Automated freight retrieval in spot markets

Within the Digital Factory, CAPE Groep offers a challenging and inspiring environment for scientific research into digital transformation and innovation. We translate strategic issues into agile solutions for various organizations. CAPE Groep works together with universities and knowledge institutions, such as the University of Twente, TKI Dinalog and NWO, continuously on innovative projects. In this way we bring science and business together and arrive at solutions that make a difference. Topics in which we offer assignments are within the areas of digital transformation, CI/CD, blockchain, BI and big data, machine learning, IoT and innovation.

We regularly publish an adapted management summary of these Thesis studies on our website.

Research: 'Design of an automated freight search application'
MSc. Industrial Engineering & Management, University of Groningen


Transport companies often have to deal with 'empty runs'. According to the European Commission, trucks drive 23% of their kilometers without freight. This can occur when, for example, a product is exported from the Netherlands to Germany, but there is no freight to take back. A truck that drives without cargo is very expensive for transport companies. Waiting until a load has been found that can be taken back is just as expensive, because the time the truck is waiting it can of course not generate any income.

In order to be able to increase transport efficiency, transport companies use online freight portals, also known as 'spot markets'. Spot markets can be used to search for available freight to be transported, or to offer available space in a truck. This prevents the costly empty trips. An ideal solution, you would think, but the use of spot markets is often not preferred by transport companies. This is because, according to the transport companies, there is a lack of:

  • Efficiency: It is time consuming and labor intensive: logistics planners have to perform many manual actions to find a cargo;
  • Effectiveness: Transport companies often operate in several spot markets to increase the chances of finding a suitable freight. These sports markets are very dynamic, which may require quick action to find suitable cargo. This high dynamics on different portals contributes to a lower effectiveness.
  • Reliability: Every transport company can register on the portals, which means that reliability cannot always be guaranteed.

Because of these disadvantages, many transport companies first try to find a freight or carrier within their own network and the spot market is often seen as the least desirable option.

The aim of my graduation project was to develop a prototype for our client of an application that makes it possible to automatically search for freight on the spot markets. The application must meet the requirements set by the customer and must be able to deliver smart data to the user, which ensures that efficiency, effectiveness and reliability increase. This will make it more attractive for transport companies to offer freight or available space on the spot market, saving costs and making transport more sustainable.

Prototype design

The process of searching for freight starts the moment the logistics planner receives a message that a truck is about to run empty. Based on this information, the planner can determine his 'input parameter', namely the location where the truck must be loaded again and the cargo unloaded. Based on this input, a web scraper is initiated that retrieves freight data from various spot markets. This data is merged and specific logic can be applied in Mendix to show only the best results to the user. The 'smart' information will lead to improved decisions by the user.


The proposed prototype design was subsequently developed on the Dexi and Mendix platforms. The result is a web scraper that retrieves the desired data from various spot markets on the basis of entered parameters. The web scraper actually mimics the behavior of a user who manually searches for freight orders. The data that is 'scraped' corresponds exactly to the data displayed on the spot markets. Since the data comes from different spot markets, it is first sorted and merged. This results in a table with the data that can be further edited in Mendix.

In Mendix 'custom logic' is then applied to make the obtained data 'smarter'. This is achieved using a decision-making model that ensures that the loads that most closely match the user's wishes are shown first.


Based on the validation with stakeholders, it can be concluded that the application can contribute to the quality of data that the logistics planners have available. The low efficiency in the current situation is solved by reducing the actions performed by the planners (navigating and searching on different spot markets) to a single search. By scraping several spot markets at the same time, this dynamic environment can be clearly displayed, which results in more suitable loads and therefore a higher effectiveness. The last core problem, low reliability, is solved with the ability to assess delivered freights or blacklist companies. As a result, only the most reliable companies are shown to the user.

Rick - business IT consultant

Rick Hendriks
Business Consultant