Cargando…
A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification
We approach the task of detecting the illicit movement of cultural heritage from a machine learning perspective by presenting a framework for detecting a known artefact in a new and unseen image. To this end, we explore the machine learning problem of instance classification for large archaeological...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Nature Publishing Group UK
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9356139/ https://www.ncbi.nlm.nih.gov/pubmed/35931710 http://dx.doi.org/10.1038/s41598-022-15965-2 |
_version_ | 1784763450309738496 |
---|---|
author | Winterbottom, Thomas Leone, Anna Al Moubayed, Noura |
author_facet | Winterbottom, Thomas Leone, Anna Al Moubayed, Noura |
author_sort | Winterbottom, Thomas |
collection | PubMed |
description | We approach the task of detecting the illicit movement of cultural heritage from a machine learning perspective by presenting a framework for detecting a known artefact in a new and unseen image. To this end, we explore the machine learning problem of instance classification for large archaeological images datasets, i.e. where each individual object (instance) is itself a class that all of the multiple images of that object belongs. We focus on a wide variety of objects in the Durham Oriental Museum with which we build a dataset with over 24,502 images of 4332 unique object instances. We experiment with state-of-the-art convolutional neural network models, the smaller variations of which are suitable for deployment on mobile applications. We find the exact object instance of a given image can be predicted from among 4332 others with ~ 72% accuracy, showing how effectively machine learning can detect a known object from a new image. We demonstrate that accuracy significantly improves as the number of images-per-object instance increases (up to ~ 83%), with an ensemble of classifiers scoring as high as 84%. We find that the correct instance is found in the top 3, 5, or 10 predictions of our best models ~ 91%, ~ 93%, or ~ 95% of the time respectively. Our findings contribute to the emerging overlap of machine learning and cultural heritage, and highlights the potential available to future applications and research. |
format | Online Article Text |
id | pubmed-9356139 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Nature Publishing Group UK |
record_format | MEDLINE/PubMed |
spelling | pubmed-93561392022-08-07 A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification Winterbottom, Thomas Leone, Anna Al Moubayed, Noura Sci Rep Article We approach the task of detecting the illicit movement of cultural heritage from a machine learning perspective by presenting a framework for detecting a known artefact in a new and unseen image. To this end, we explore the machine learning problem of instance classification for large archaeological images datasets, i.e. where each individual object (instance) is itself a class that all of the multiple images of that object belongs. We focus on a wide variety of objects in the Durham Oriental Museum with which we build a dataset with over 24,502 images of 4332 unique object instances. We experiment with state-of-the-art convolutional neural network models, the smaller variations of which are suitable for deployment on mobile applications. We find the exact object instance of a given image can be predicted from among 4332 others with ~ 72% accuracy, showing how effectively machine learning can detect a known object from a new image. We demonstrate that accuracy significantly improves as the number of images-per-object instance increases (up to ~ 83%), with an ensemble of classifiers scoring as high as 84%. We find that the correct instance is found in the top 3, 5, or 10 predictions of our best models ~ 91%, ~ 93%, or ~ 95% of the time respectively. Our findings contribute to the emerging overlap of machine learning and cultural heritage, and highlights the potential available to future applications and research. Nature Publishing Group UK 2022-08-05 /pmc/articles/PMC9356139/ /pubmed/35931710 http://dx.doi.org/10.1038/s41598-022-15965-2 Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Article Winterbottom, Thomas Leone, Anna Al Moubayed, Noura A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title | A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title_full | A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title_fullStr | A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title_full_unstemmed | A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title_short | A deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
title_sort | deep learning approach to fight illicit trafficking of antiquities using artefact instance classification |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9356139/ https://www.ncbi.nlm.nih.gov/pubmed/35931710 http://dx.doi.org/10.1038/s41598-022-15965-2 |
work_keys_str_mv | AT winterbottomthomas adeeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification AT leoneanna adeeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification AT almoubayednoura adeeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification AT winterbottomthomas deeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification AT leoneanna deeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification AT almoubayednoura deeplearningapproachtofightillicittraffickingofantiquitiesusingartefactinstanceclassification |