In the summer vacation of 2013, there was about a month left before the start of the competition.

"The training process of the model requires all weights, data and many intermediate processes to be put into the GPU for processing. Therefore, the size of the GPU memory is particularly important." Meng Fanqi sighed, "Even the flagship 690 we purchased is too small.

It’s only 4G in size.”

Compared with the A100-80G, which was later banned by the United States from being sold to China, the 690 has 20 times less video memory alone, not to mention other performance. Now Meng Fanqi can only iterate the model pitifully using 16 pictures at a time.

"Sixteen pictures at a time, one cycle requires close to one million times to update the entire data set. And if you want to converge the model well, hundreds of cycles are indispensable."

Meng Fanqi estimated that it would take nearly 20 days for this version to produce a result. The final training process indeed took about three weeks to converge to the current performance.

Fortunately, IMAGE has basically become a training data set that every algorithm engineer must adjust. Meng Fanqi himself has ranked on the list countless times, so he is naturally familiar with it and knows the approximate settings of various parameters.

This saved him at least a month or two of precious time.

Even though a training session takes three weeks, Meng Fanqi still prepared a version of the model before the competition started.

Seeing that the final performance of the trained model met expectations, Meng Fanqi finally felt a big stone in his heart.

In the past few months, the only thing he has been worried about is that the old framework from many years ago will have some unexpected problems, causing the final results to not match theoretical expectations.

Once this happens, the cost of finding the problem and testing the solution will be too high. If it cannot be solved in time, it will greatly affect his initial planning.

The current result is about a top-5 error rate of 4.9%. This version is slightly worse than the performance in later papers, but fortunately it is still better than the human standard given by the competition.

Generally speaking, the specific data used in the competition will not be announced before the competition. It is just that the IMAGE competition is special. There are more than 10 million images. It is impossible to throw them away and no longer use them after one or two competitions.<

/p>

Therefore, the data used in each competition changes very little, but the specific track, competition content and judging methods are often adjusted.

Although IMAGE can actually submit results during the offseason, and Meng Fanqi can upload the results now and win the first place, the attention after all cannot be compared with the fierce competition during the game.

At the same time, Don Juan finally began to realize that the direction of things had deviated far from what he expected.

"I remember I found that Alex's accuracy on this was less than 85, but now yours is over 95." Don Juan couldn't believe this fact when he came to check the results for the first time.

"Are you sure you're not mistaken? Don't fool me, brother. If you don't read enough, you can easily be deceived." Don Juan's mentality at the moment was very complicated. He really hoped that this was true, but because things seemed too good, he was very confused.

Hard to believe.

"It's fake, I lied to you." Meng Fanqi rolled his eyes, "I added special effects, and they are all chemical ingredients."

"No way, I have seen with my own eyes that this performance has improved along the way." Don Juan flipped through the model training log again, with a hint of grievance in his voice. He had just been thinking about it and hugged his thighs tightly and walked up.

A scene at the pinnacle of life.

This is a poor person who is worried about gains and losses. He can’t believe it, but he is afraid that it is fake.

"Although I don't have the real answers to the test set, I cut out 5% of the training set and didn't use it as a verification method." Meng Fanqi can be said to have a clear understanding of the variance of this data set, 95% of it.

The data is used for training and 5% of the data is used for testing, which is a very safe and conservative ratio.

"In other words, as long as the 5% data is not much different from the test set data, your method can be ten percentage points better than last year's champion?" Don Juan was still in extreme shock. "It's that simple.

?You all fell down before I even tried my best?"

Don Juan's feeling at this time was like Ya Shenyue's first discovery that he could directly assign the God of Death to get rid of his biggest opponent L. The efforts, hard work and struggle he imagined did not happen, and it was completely unnecessary. It was shocking.

Achievements and progress are made even before the game officially begins.

"This is life. Success or failure may have nothing to do with you in many cases. Just get used to it." Meng Fanqi patted his shoulder, "It's okay if you don't get used to it this time. There is still a long road ahead. You will

Get used to it."

Because there’s nothing you can do if you’re not used to it, right? People who can’t change their weight can only change their aesthetics.

Otherwise, you will be tortured by yourself for the rest of your life.

Now that we have achieved this result on 95% of the data, the next thing to do is to add the remaining 5% and continue to fine-tune the model for a few days.

In this way, the final results can be used directly for submission in November.

If we continue to fine-tune the performance of a model that already performs quite well, it will take far less than 21 days.

It only takes about two days for the new training log to show that the model's performance has basically converged to a fixed value and rarely continues to fluctuate.

In this case, Meng Fanqi has only one thing left to do before going to the Australian conference, and that is to complete the experimental data of these papers at hand.

Fill in the last missing piece of the puzzle in these articles.

As of this time, Meng Fanqi has completed approximately 7 articles. In addition to the core of this competition, the new model Dream based on residual ideas, as well as related training techniques, batch normalization, Adam second-order optimizer,

and Mix-up data enhancement.

Meng Fanqi also prepared groundbreaking work in three other directions to capture three key areas.

Among the relevant content in the competition, only the residual network can actually be regarded as groundbreaking content. The remaining three, although they are masterpieces in their respective directions, can hardly be regarded as the foundation works of a certain subdivision.<

/p>

Writing a paper to describe it in detail was just a matter of helplessness, because in order to ensure the performance and training speed of Dream, Meng Fanqi had no choice but to use some techniques.

In order to ensure that such important results could be replicated in the industry, Meng Fanqi had to describe these training techniques in detail, so he wrote a paper. But if he had the choice, he would not rush it.

What he really hopes to seize the opportunity to deploy is, first, the generative adversarial network, which he had previously discussed with Dean Fu. This is the most promising and elegant unlabeled learning method in recent years, and is the future of all generative networks.

A milestone that is difficult to avoid for similar technologies.

Second, it is a real-time detection network based on new ideas. This will greatly improve the speed and accuracy of distinguishing objects and determining their position in pictures. It will be the most widely used image detection technology in the future, whether it is face recognition, autonomous driving or industry.

Detection. These new technologies have to mention the importance of this speed increase.

Third, it is the most concise and easy-to-use segmentation network, U-. This will be the baseline for complex segmentation tasks and will dominate the field of medical images.

Meng Fanqi selected these three categories and added the residual network, which covers the four major fields of classification, detection, segmentation and generation. It occupies the four main tracks of image algorithms.
Chapter completed!