[Server-side behind-the-scenes story] Interview | Rewrite IgnisMemoria engineer's honest thoughts
This is Nakauchi from the Technical Sales Department
We had a talk about the server side of "Rewrite IgnisMemoria" (hereinafter referred to as RewriteIM), which Visual Arts released on February 6, 2017
The second half of the event will be a dialogue between Visual Arts engineer Kouki Yamada and our engineer Ryosuke Sashihara
↑Visual Arts engineer Kouki Yamada
On Twitter, I reply to all my followers
- Thank you for joining us today. First of all, please tell us what makes Visual Arts unique
The company has been around for over 20 years, originally starting out with PC games, then creating the Key
Originally, I was making games, but then I started creating game engines and running a franchise business that rents them out to other companies
In terms of content, we support development companies that use game engines, so that the development companies can focus on making games, while we take care of sales and running the games. That's our stance
- When did you start playing social games?
As for social games, I started getting into them about 4 or 5 years ago, and my first game was "Little Busters! Card Mission," which I released on DeNA's mobage
This game was created by Soichiro Fuki, the producer of the current game, RewriteIM, as director. It all started here
The second game was "Angel Beats!" and this RewriteIM is the third
- Has team Aeca been around since you first started making social games?
No, it wasn't there from the beginning.
At first it was called the Social Team.
There wasn't a specific team name, but a team was formed to make a social game.
After that, another team was formed and created "Angel Beats!"
However, there was some dissatisfaction from the field that "it's impossible to run the business with such a small number of people," so we
decided to go ahead and do it as one team, and that's how "team Aeca" was born.
- How many members are there on team Aeca?
There are 9 people
- 9 people!? That's not many..
Yes, there aren't many!!
After all, I'm just one person running the server (lol).
The person on Twitter
- Twitter too?!
Speaking of which, you reply to all your followers on Twitter, but isn't that a lot of work?
Yes, it's tough (laughs).
It's tough, but it's something we do with our users in mind.
I think it's quite nice to get an official response.
So I try to reply to messages from users as much as possible, and I've been doing this since the days of "Angel Beats!"
- Visual Arts has many popular IPs, but
do you plan to continue releasing social games in the future?
That's right. There's the possibility of "Charlotte," which was broadcast as a TV anime in 2015, or perhaps "team Aeca" could launch a new IP
It hasn't been decided yet!
- I'd like to ask you a question about yourself. What made you decide to become an infrastructure engineer?
My previous job was at an independent IT company, and before that I was a freelancer, but..
- Huh? Did you start out as a freelancer?
Yes, that's right. It's rare, isn't it? (laughs)
After working as a freelancer, I started working for an independent IT company, but it wasn't much fun, so I quit. After about a year of going back to being a freelancer, I thought it might be interesting to work at a different company, so I came to this company
When I first joined the company, I was creating websites and web systems, and didn't do any infrastructure work at all.
It was actually with the previous game, "Angel Beats!" that I started working on infrastructure.
Since we are not originally a social game company, we do not have any infrastructure engineers or network engineers
Anyway, make sure the server doesn't go down
- What did you pay attention to when developing the server side this time?
What we paid attention to was load balancing and making sure the server didn't go down.
We also wanted to do something to balance the load so that we could withstand any issues.
I couldn't do that with "Angel Beats!"
We started with six cars (laughs)
- That would kill you
I didn't even know how to balance the load
We were just doing things like this for the web, this for the DB, this for memcached, etc., but we had so many users that we
just couldn't keep up.
- Did you come here instead of RewriteIM?
RewriteIM is coming
- Even so, with six machines and both the web and database, it's possible that things will die
Yes, you'll die
We started with three web servers, two database servers, and one memcached server, but of course they all crashed, and the web server died first, so we decided we needed to add more web servers, and ended up adding about 30
Because we increased it too much, the number of connections increased too much and we were no longer able to connect to memcached, and eventually the database crashed
However, at that time I hadn't even considered distributing the database, so I set aside a one-month maintenance period and rebuilt everything
- After rebuilding, how many servers did you end up with?
In the end, there were about 80
It was good that we were able to carry out the load test to that extent
- What were some of the difficulties you personally faced this time?
Yamada : It's a load test.
Sashihara: That almost broke my heart
Yamada: I think the database is pretty solid this time..
Sashihara: That's true. But I wonder why it falls
Yamada: That's true. In the end, the problem of crashing under heavy loads hasn't been solved yet
Sashihara: So it was a pretty close call and a scary release, but
I'm glad we managed to pull it off.
Yamada: Right now (as of March 2, 2017, when the interview was conducted), we have around 800,000 users, but our previous game, "Angel Beats!", had around 600,000 users after two years of operation, so we've already surpassed that number even though it hasn't even been a month
Sashihara: When we first released it, I was also counting the number of users in the database. The
number of users was increasing at a tremendous rate, so I was worried about whether it would be okay.
Yamada: If anything, it was the CDN for downloads that was in a terrible state
Sashihara: The amount of traffic on the CDN was enormous
Yamada: It's about 400TB now
Sashihara: That's about it
Yamada: So I asked our engineers to somehow reduce the number of downloads, but
of course it's impossible to reduce it.
Sashihara: The initial downloads were quite large
Yamada: It's big. It's about 580MB
Sashihara: It goes to the CDN, so it goes that far
Yamada: But going back to what we were talking about, the most difficult part was the load testing. It was really tough.
But I'm glad we were able to carry out that much load testing.
Sashihara: Thank you for adjusting your shifts to coincide with my night shift
Yamada: Actually, we didn't meet up, we were just at the office (laughs)
Sashihara: When I did it with Buttoushi, I was a little distracted (laughs)
Yamada: We stayed up all night (laughs)
Sashihara: On the weekends before the release..
Yamada: I lived in the company!
Sashihara: I also felt like going home to take a bath (laughs)
Yamada: There's a public bath near my house that opens at 10am, so I always went there
-Finally, please tell us what you thought about using Beyond's services
I feel so grateful that they have been so considerate to me
And the price is cheap!!
The biggest thing was that I was able to sleep without anything happening. I
was able to sleep and no phone calls came in the middle of the night, so
I no longer had to go to work on my PC at home after getting a call at around 5:00 in the morning, which was really good.
- That's all for the interview. Thank you for today
