[Laravel] What does "Maximum retries of 10000 reached without finding a unique value" mean? [Faker]

Hello.
I'm Mandai, the Wild team member in charge of development.

, which is included by default Faker you'll most likely use
It's convenient because it supports multiple languages ​​and can create various types of test data, but it also has a very useful method called `unique` that's helpful when creating multiple data with unique IDs.

However, depending on how you use it, you may get an error like the one in the title, and you may not know what it means, or you may not be able to resolve it and have to try a different method... If that happens, it may seem like a waste of having installed Faker! So, this time I would like to introduce how to resolve the error "Maximum retries of 10,000 reached without finding a unique value" that is often seen with Faker

 

Cause of the error

Judging from the error message, I suspect I've run into an infinite loop somewhere, but I have almost no idea what it might be. All I know is that the error seems to be occurring around the time I'm using Faker's unique() method. I'm
not sure I've written any code that would cause an infinite loop.

Resetting it at the right time makes it work properly, but of course, resetting it will cause duplicates. Finding the right
balance is something you'll only understand after using it for a while, but once you figure out the cause, the solution should become clear naturally...!!

So, let's take a look at the code for Faker's unique() method

 

Tracking unique() behavior

This part of the GitHub repositoryis the code for the `unique()` method.
It passes the received variable directly to a class called `UniqueGenerator`, so let's take a look at the code for the `UniqueGenerator` class.

The behavior is cleverly controlled by a magic method, but essentially, it's just calling a method of the Faker\Generator object received when the object is created with UniqueGenerator::__call().
The while condition checks for duplicates, so the returned value is unique.
Checking for duplicates using array keys seems rather primitive, but the cause is easy to understand.
The error message when an OverflowException is thrown contains the wording in the title, and the cause was that the number of iterations of this do-while statement reached the retry limit.

 

How to avoid errors

There is only one adjustment you can make to avoid the error: change the value of the second argument of the unique() method and increase the number of loop retries.
The default is 10,000 retries, but if you want 15,000 values, it will definitely overflow.

To increase the number of retries:

$factory->define(User::class, function(Faker\Generator $faker) { return [ 'name' => $faker->name(), 'age' => $faker->unique(false, 15000)->numberBetween(1, 80), // Change the number of retries to 15000 ]; });

 

Another possible cause is that the number of variations in values ​​returned by the Faker\Generator object is less than 10,000.
As in the example above, since numberBetween() only returns values ​​from 1 to 80, a unique value cannot be generated for the 81st data point, and the loop will execute beyond the number of retries.
While names and other random combinations seem to offer a wide variety, there are only about 3,000 English first names and 473 English last names, so there is a fair amount of variation. However, in Japanese, there are only about 50 given names and 31 surnames, and even full names only have about 1,500 variations.

Even if you want the full names of 5000 people in Japanese, if there are only 1500 possible combinations, then using `unique()` would be pointless, and you'd need to think of another method.
For example, it seems there's a way to implement and add a new provider to the `Faker\Generator` class, but I haven't looked into it in detail yet, so I hope to introduce it sometime in the future.

 

summary

This time, I looked into the error "Maximum retries of 10000 reached without finding a unique value," which I frequently encounter when using Faker.
It all started when I looked up the error message online and came across something on StackOverflow that basically said, "Just use reset(true)!" I thought that was completely ridiculous.

I wondered if there was some fundamental misunderstanding, so I looked into it, but it seems there was more than one pitfall

If you come across the message "Maximum retries of 10,000 reached without finding a unique value," investigate the number of retries and the amount of dummy data obtained

 
That's all