This project is from team Amaris, which was awarded the Third Prize in the Alibaba Cloud Singapore AI Hackathon.
Our project is a fully functioning Android application that takes in an image of a room or a dormitory bunk, and uses multimodal LLMs to determine the hygiene and cleanliness rate based on the image, as well as provide a platform for users to ask questions to an AI chatbot with contents related to the official published guidelines.
The main value of our project is to improve the living conditions of 1.4 million (and growing!) foreign workers by enforcing the new FEDA expanded guidelines defined by the Ministry of Manpower, Singapore. This new guideline will be slowly rolled out and enforced by the year 2030. We also aim to assist dormitory workers and their managers in understanding the guidelines.
Currently, the FEDA guideline which covers strict hygiene standards for dormitories, hostels, et cetera, is only applicable to the 300-400 largest dormitories in Singapore. However, the guidelines were changed last year April 2023 to include all 1.6k dormitories under the same strict standard by 2030. Hence this is where the pain point is as the dormitory inspectors will now have 4-5x more workload.
Dorm inspectors in Singapore face the daunting task of manually checking the cleanliness of approximately 1,600 dormitories accommodating around 1.4 million migrant workers. However, with the innovative DormChecker solution, each migrant worker now has a tool on their phone to simplify the process. By snapping a photo of their living conditions, they can swiftly assess the cleanliness and hygiene score of their dormitory. Additionally, uncertainty can be addressed through Protocol AI, a chatbot integrated into the system, providing education on cleanliness standards and ensuring a more efficient and proactive approach to maintaining a healthy living environment for migrant workers.
Our project leverages 3 different technologies from Alibaba Cloud, namely OpenSearch, Elastic Compute Service (ECS), and Tongyi Qianwen (Qwen) Vision Language Chat (Qwen-VL-Chat) model.
Firstly, we used OpenSearch to ingest the documents. This is so that the LLM chat interface can leverage Retrieval-Augmented Generation, also known as RAG, to generate very accurate answers. Our envisioned use case for this is to educate the dormitory workers with regard to the new guidelines. OpenSearch is also scalable as we can ingest new documents as and when they are published.
Next, we chose to use ECS because it is very flexible and scalable, allowing us to run multiple services on the same instance. It has a wide range of hardware options available and we found that it suits our needs best. We have tried to use PAI-EAS with the guidance of the mentors, as it would supposedly be faster to deploy, but unfortunately, it does not support Qwen-VL-Chat by default. We have also attempted to upload and run the model via Object Storage Service (OSS), but PAI-EAS does not support the Transformers version that Qwen-VL-Chat requires to run.
Lastly, we chose to use Qwen-VL-Chat as it is able to determine the contents of images with very high accuracy. With some prompt engineering, we are able to get reliable results most, if not all the time.
My name is Nicholas Koh Wei Xuan and I am a Software Engineer at Amaris AI Pte Ltd. My team consists of myself, Raina Tang, Hakim Razalee, Alam Sah, and we come from the same company.
Amaris AI is a cutting-edge technology company specializing in artificial intelligence solutions that empower businesses to thrive in the digital era. With a passion for pushing the boundaries of what's possible, we harness the power of AI to drive efficiency, enhance decision-making, and unlock new opportunities across various industries. Our dedicated team of experts, fueled by creativity and expertise, collaborates seamlessly to deliver tailored AI solutions that address the unique challenges of our clients. At Amaris AI, we believe in the transformative potential of AI to shape a smarter, more connected future.
AI Xin: Streamlining Workflows with Robotic Process Automation
113 posts | 24 followers
FollowAlibaba Cloud Community - March 27, 2024
AsiaStar Focus - June 30, 2022
Alibaba Cloud Community - March 28, 2022
Alibaba Clouder - March 16, 2018
Alibaba Cloud Community - January 4, 2024
Alibaba Clouder - December 1, 2020
113 posts | 24 followers
Follow
Offline Visual Intelligence Software Packages
Offline SDKs for visual production, such as image segmentation, video segmentation, and character recognition, based on deep learning technologies developed by Alibaba Cloud.
Learn More
AI Acceleration Solution
Accelerate AI-driven business and AI model training and inference with Alibaba Cloud GPU technology
Learn More
Tongyi Qianwen (Qwen)
Top-performance foundation models from Alibaba Cloud
Learn More
1688 Cloud Hub Solution
Alibaba Cloud 1688 Cloud Hub is a cloud-based solution that allows you to easily interconnect your 1688.com store with your backend IT systems across different geographic regions in a secure, data-driven, and automated approach.
Learn MoreMore Posts by Alibaba Cloud Project Hub
