‘Small Data' Are Also Crucial for Machine Learning
Many people relate "artificial intelligence" with "big data. "There's a reason for that: some of the most prominent AI breakthroughs in the past decade have relied on enormous data sets. Image 1 made great progress in the 2010s thanks to the development of ImageNet, a data set containing millions of images hand sorted into thousands of categories. More recently, GPT-3, a language model, was trained on 2 online texts to produce humanlike text in Jan,
2021.So it is not surprising to see AI being tightly connected with "big data" in the 3 imagination. But AI is not only about large data sets, and research in "small data" approaches has grown extensively over the past decade. The so-called transfer learning serves as an especially 4 example.
Also known as "fine-tuning," transfer learning is helpful in settings where you have 5 data on the task of interest but abundant data on a related problem. You need to first train a model using a big data set and then retrain slightly using a smaller one related to your 6 problem. A research team working on German-language speech recognition, 7 , showed that they could improve their results by starting with an English-language speech model trained on a larger data set. Then, they used transfer learning to 8 that model for a smaller data set of German-language audio.
Small data approaches such as transfer learning are more 9 than more data-intensive methods. They can promote progress in areas where little or no data exist, such as in forecasting natural hazards that occur relatively 10 .In this context, small data approaches will become increasingly important as more organizations look to diversify AI application areas and invest in previously 11 fields.
Despite the progress in research, transfer learning has received relatively little 12 .While many machine learning experts are likely familiar with it at this point, the existence of techniques such as transfer learning does not seem to have reached the awareness of the broader space of policymakers in positions of making important decisions about AI funding and 13 .
As long as the success of small data technique like transfer learning is 14 ,resources can be allocated to support their widespread use. In that case, we can help correct the popular 15 regarding the role of data in AI and foster innovation in new directions.
