This study presents an innovative framework to improve the accessibility and usability of collaborative robot programming. Building on previous research that evaluated the feasibility of using a domain-specific language based on behaviour-driven development, this paper addresses the limitations of earlier work by integrating additional features like a drag-and-drop Blockly web interface. The system enables end users to define and execute robot actions with minimal technical knowledge, making it more adaptable and intuitive. Additionally, a gesture-recognition module facilitates multimodal interaction, allowing users to control robots through natural gestures. The system was evaluated through a user study involving participants with varying levels of professional experience and little to no programming background. Results indicate significant improvements in user satisfaction, with the system usability scale overall score increasing from 7.50 to 8.67 out of a maximum of 10 and integration ratings rising from 4.42 to 4.58 out of 5. Participants completed tasks using a manageable number of blocks (5 to 8) and reported low frustration levels (mean: 8.75 out of 100) alongside moderate mental demand (mean: 38.33 out of 100). These findings demonstrate the tool's effectiveness in reducing cognitive load, enhancing user engagement and supporting intuitive, efficient programming of collaborative robots for industrial applications.

In this paper, the author presented the challenges for autonomous system in indoor climate-controlled environments from the point of view of system integration, monitoring and optimization of energy storage for application of autonomous environment monitoring. In first instance, we proposed the comparison of two different deep learning algorithms for fire detection in order to enable a micro-arial vehicle to automatically detect the fire areas. On the other hand, due a range of limitations (e.g., battery, power computation, etc.) based on the previous experimental analysis presented, an intelligent battery control for an integrated local renewable energy for a climate-controlled greenhouse is presented and verified. Based on the experimental results of the proposed intelligent control strategy, the feasibility and economical cost was verified for an on-grid renewable photovoltaic system with battery energy storages. 

As robots increasingly integrate into the workplace, Human-Robot Collaboration (HRC) has become increasingly important. However, most HRC solutions are based on pre-programmed tasks and use fixed safety parameters, which keeps humans out of the loop. To overcome this, HRC solutions that can easily adapt to human preferences during the operation as well as their safety precautions considering the familiarity with robots are necessary. In this paper, we introduce GPTAlly, a novel safety-oriented system for HRC that leverages the emerging capabilities of Large Language Models (LLMs). GPTAlly uses LLMs to 1) infer users’ subjective safety perceptions to modify the parameters of a Safety Index algorithm; 2) decide on subsequent actions when the robot stops to prevent unwanted collisions; and 3) re-shape the robot arm trajectories based on user instructions. We subjectively evaluate the robot’s behavior by comparing the safety perception of GPT-4 to the participants. We also evaluate the accuracy of natural language-based robot programming of decision-making requests. The results show that GPTAlly infers safety perception similarly to humans, and achieves an average of 80% of accuracy in decision-making, with few instances under 50%. Code available at: https://axtiop.github.io/GPTAlly

Foundation models provide the adaptability needed in robotics but often require explicit tasks or human verification due to potential unreliability in their responses, complicating human-robot collaboration (HRC). To enhance the reliability of such task-planning systems, we propose 1) an adaptive task-planning system for HRC that reliably performs non-predefined tasks implicitly instructed through HRC, and 2) an integrated system combining multimodal large language model (LLM)-based task planning with multimodal communication of human intention to increase the HRC success rate and comfort. The proposed system integrates GPT-4V for adaptive task planning and comprehension evaluation during HRC with multimodal communication of human intention through speech and deictic gestures. Four pick-and-place tasks of gradually increasing difficulty were used in three experiments, each evaluating a key aspect of the proposed system: task planning, comprehension evaluation, and multimodal communication. The quantitative results show that the proposed system can interpret implicitly instructed tabletop pick-and-place tasks through HRC, providing the next object to pick and the correct position to place it, achieving a mean success rate of 0.80. Additionally, the system can evaluate its comprehension of three of the four tasks with an average precision of 0.87. The qualitative results show that multimodal communication not only significantly enhances the success rate but also the feelings of trust and control, willingness to use again, and sense of collaboration during HRC. 

Greenhouse cultivation offers the advantage of controlled growth conditions, leading to enhanced crop productivity and quality. However, maintaining these optimal conditions requires substantial energy, resulting in increased greenhouse gas emissions and operational costs. Integrating local renewable energy sources, particularly photovoltaic (PV) solar energy, has demonstrated the potential to reduce energy consumption and costs. This paper proposes a machine learning-based intelligent control strategy for greenhouses using a solar photovoltaic system combined with battery energy storage system (BESS). Long short-term memory (LSTM) forecasting models are developed to forecast power consumption and solar PV production. The outputs of these forecasting models are then fed into a reinforcement learning (RL)-based battery control model. This control model is trained to minimize energy costs by taking advantage of variable electricity spot prices and reducing peak energy consumption. The performance of the proposed system is evaluated with actual operational data. Results show that the proposed model reduces variable charges by 2.2% and 2.7% and decreases peak energy consumption by 24% and 19% in February and March 2023, respectively. Overall, the proposed system demonstrated the potential to utilize local renewable energy, minimize operational costs, and achieve a sustainable energy balance. 

Our research aims to develop an intelligent control system for optimizing the operation of lighting systems for a jogging track in the countryside by using adaptive control methods, and optimization of built-in lighting control. Through this, lighting systems will be optimized, energy consumption will be minimized and the lighting system will be adapted to be able to handle a larger amount of local renewable energy production. Due to the level of complexity, in this paper, we have focused to integrate a commercial lighting control system and the solar energy system with energy storage. In particular, their respective components were selected and integrated by means of a distributed control architecture. An application programming interface (API) was developed by using the design science research methodology aiming at bridging proprietary systems for sustainable lighting solutions. In addition, a series of focus group interviews were conducted to give input to the optimization of the lighting system from a user perspective. 

Our research aims to develop an intelligent control system for optimizing the operation of lighting systems for a jogging track in the countryside by using adaptive control methods, and optimization of built-in lighting control. Through this, lighting systems will be optimized, energy consumption will be minimized and the lighting system will be adapted to be able to handle a larger amount of local renewable energy production. Due to the level of complexity, in this paper, we have focused to integrate a commercial lighting control system and the solar energy system with energy storage. In particular, their respective components were selected and integrated by means of a distributed control architecture. An API was developed by using the design science research methodology aiming at bridging proprietary systems for sustainable lighting solutions. 

In this paper we introduce an end-user programming approach for defining interaction sequences with collaborative robots using gestures. The approach relies on a textual domain-specific language (DSL) to allow end users to specify the expected robot behaviour through Behaviour-Driven Development (BDD) scenarios. To evaluate the feasibility of our approach, we conducted a user study with 12 participants having an entry level in programming. The participants were asked to code a sequence in which the collaborative robot responds to their body gestures by performing a pick-and-place task. The study’s findings indicate that the method is both effective and user-friendly for individuals with limited programming experience, while highlighting opportunities for adopting new end-user development practices.