VLSP2021 vieCap4H Challenge: Legacy Version

Organized by viecap4h-organizers - Current server time: April 20, 2024, 4:05 a.m. UTC

Previous

Public Test
Sept. 24, 2021, 5 p.m. UTC

Current

Private Test
Oct. 19, 2021, 5 p.m. UTC

End

Competition Ends
Never

Our website:

https://vlsp.org.vn/vlsp2021/eval/vieCap4H

 

LEGACY version 

  • The original competition of VLSP2021 was closed.
  • Here we create this legacy version for participants who wish to continue their experiments on our Public & Private round datasets. 

 

Important dates:

  • Aug 5, 2021: Registration open
  • Aug 30, 2021: Registration closed
  • Sep 20, 2021: Challenge started (via AIHUB.VN)
  • Sep 25, 2021: Public testing phase started
  • Oct 15, 2021: Registration deadline for using pre-trained models. 
  • Oct 20, 2021: (1) Private testing phase started, (2) deadline for team merging, and (3) deadline to sign USER AGREEMENT form [Link] and send back to us via email [email protected].
  • Oct 25, 2021: Private testing phase ended at 23:59:59 GMT+7.
  • Oct 27, 2021: Announce selected teams to submit technical reports.
  • Nov 10, 2021: Deadline for selected teams to submit technical reports.
  • Nov 20, 2021: Final winners announcement.
  • Nov 26, 2021: Result presentation and award ceremony (workshop day).

 

Introduction

Humans are unique in their capability to interpret and describe their visual perception in natural language. Although modern AI has achieved ground-breaking successes in the last decade, building a machine that learns to talk about what it sees remains very challenging. In this playground, Image Captioning, a machine learning task to automatically generate natural language descriptions of a given image, has emerged and attracted enormous attention in the AI research community. The task is fascinating and yet challenging at the same time as it sits on the bridge between Computer Vision and Natural Language Processing, the two most important fields of modern AI. 

The COVID-19 pandemic has exacerbated the ongoing shortage of health workers globally, posing an urgent need for smart assistants that can effectively cooperate with humans to fill the gap. Towards this ultimate goal, this challenge aims at assessing the machine’s ability to use Vietnamese to describe the visual content in healthcare settings. It provides the participants an opportunity to contribute their knowledge to advance the field and make potential applications of the task in either healthcare settings and general settings (e.g. virtual assistants for blind and visually impaired people, or visual content indexing and searching) accessible for the local community.

 

Contact Us:

Please feel free to contact us if you have any questions (privately) [email protected] or publicly at https://groups.google.com/g/viecap4h-organizers

 

References:

[1] X. Chen, H. Fang, TY Lin, R. Vedantam, S. Gupta, P. Doll ́ar, and C. L. Zitnick. 2015. Microsoft coco captions: Data collection and evaluation server.

[2] Xuan-Son Vu, Thanh-Son Nguyen, Duc-Trong Le, Lili Jiang, Multimodal Review Generation with Privacy and Fairness Awareness, In: Proceedings of the 28th International Conference on Computational Linguistics (COLING), 2020.

 

Result submission 

To evaluate results, please submit a JSON file which contains a generated caption for each image in the test file. The order of image ids in submission files must be in the same order as in the provided test file. The submission format should be the following:

 

[{“id”: “uuid_img1”, “captions”: “corresponding_caption1”}, {“id”: “uuid_img2”, “captions”: “corresponding_caption2”}]

 

Evaluation Metric

The submission will be evaluated using BLEU scores against groundtruths. In particular, we use the average score of BLEU-1, BLEU-2, BLEU-3 and BLEU-4 as the evaluation metric for image captioning generation. Please refer to NLTK’s BLEU score implementation at nltk.translate package — NLTK 3.2.4 documentation (huihoo.com) for reference.

 

General rules

  • Right to cancel, modify, or disqualify. The Competition Organizer reserves the right at its sole discretion to terminate, modify, or suspend the competition.

  • By submitting results to this competition, you consent to the public release of your scores at the Competition workshop and in the associated proceedings, at the task organizers' discretion. Scores may include but are not limited to, automatic and manual quantitative judgments, qualitative judgments, and such other metrics as the task organizers see fit. You accept that the ultimate decision of metric choice and score value is that of the task organizers.

  • By joining the competition, you accepted to the terms and conditions of Corpus User Agreement form of vieCap4H data challenge at https://bit.ly/vieCap4H-user-agreement. It is noted that your participant rights will be revoked if you do not sign and send back to us before the deadline.
  • By joining the competition, you affirm and acknowledge that you agree to comply with applicable laws and regulations, and you may not infringe upon any copyrights, intellectual property, or patent of another party for the software you develop in the course of the competition, and will not breach of any applicable laws and regulations related to export control and data privacy and protection.

  • Prizes are subject to the Competition Organizer’s review and verification of the entrant’s eligibility and compliance with these rules as well as the compliance of the winning submissions with the submission requirements.

  • Participants grant to the Competition Organizer the right to use your winning submissions and the source code and data created for and used to generate the submission for any purpose whatsoever and without further approval.

  • [New] If you wish to use any pre-trained models, please register them with detailed information with the organizers by October 10, 2021. We encourage all the participants to have a look at this thread for details of how to register your pre-trained models.

Eligibility

  • Each participant must create a CodaLab account to submit their solution for the competition. Only one account per user is allowed.

  • The competition is public, but the Competition Organizer may elect to disallow participation according to its own considerations.

  • The Competition Organizer reserves the right to disqualify any entrant from the competition if, in the Competition Organizer’s sole discretion, it reasonably believes that the entrant has attempted to undermine the legitimate operation of the competition through cheating, deception, or other unfair playing practices.

Team

  • Participants are allowed to form teams. The maximum of the number of participants on the team is up to 5. 

  • You may not participate in more than one team. Each team member must be a single individual operating a separate CodaLab account. 

  • Team mergers are allowed and can be performed by the team leader. Team merger requests will not be permitted after the "Team merger deadline".  

  • In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed for a single team as of the merge date. The maximum allowed is the number of submissions per day per phase multiplied by the number of days the competition has been running. 

  • The organizers don’t provide any assistance regarding team mergers.

 

Submission

  • Maximum number of submissions in each phase:

    • Phase 1 - Public Test: 10 submissions / day / team
    • Phase 2 - Private Test: 1 submissions / day / team
  • Submissions are void if they are in whole or part illegible, incomplete, damaged, altered, counterfeit, obtained through fraudulent means, or late. The Competition Organizer reserves the right, in its sole discretion, to disqualify any entrant who makes a submission that does not adhere to all requirements.

Data

By downloading or by accessing the data provided by the Competition Organizer in any manner you agree to the following terms:

  • You will not distribute the data except for the purpose of non-commercial and academic-research.

  • You will not distribute, copy, reproduce, disclose, assign, sublicense, embed, host, transfer, sell, trade, or resell any portion of the data provided by the Competition Organizer to any third party for any purpose.

  • The data must not be used for providing surveillance, analyses or research that isolates a group of individuals or any single individual for any unlawful or discriminatory purpose.

  • You accept full responsibility for your use of the data and shall defend and indemnify the Competition Organizer, against any and all claims arising from your use of the data.

 
No files have been added for this competition yet.

Public Test

Start: Sept. 24, 2021, 5 p.m.

Private Test

Start: Oct. 19, 2021, 5 p.m.

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 bossUIT 0.340
2 duyuit 0.316
3 thamnt2611 0.307