[Server-side behind-the-scenes story] Interview | Rewrite IgnisMemoria engineer's honest thoughts
This is Nakauchi from the Technical Sales Department
We had a talk about the server side of "Rewrite IgnisMemoria" (hereinafter referred to as RewriteIM), which Visual Arts released on February 6, 2017
The second half of the event will be a dialogue between Visual Arts engineer Kouki Yamada and our engineer Ryosuke Sashihara
↑Visual Arts engineer Kouki Yamada
On Twitter, I reply to all my followers
- Thank you for joining us today. First of all, please tell us what makes Visual Arts unique
Our company has been around for over 20 years. We originally started with PC games, and laterthe Keybrand was created. We started making games in the genre known as "tearjerker games," and that's become our basic model.
Originally, I was making games, but then I started creating game engines and running a franchise business that rents them out to other companies
In terms of content, we support development companies that use game engines, so that the development companies can focus on making games, while we take care of sales and running the games. That's our stance
- When did you start playing social games?
As for social games, I started getting into them about 4 or 5 years ago, and my first game was "Little Busters! Card Mission," which I released on DeNA's mobage
This game was created by Soichiro Fuki, the producer of the current game, RewriteIM, as director. It all started here
The second game was "Angel Beats!" and this RewriteIM is the third
- Has team Aeca been around since you first started making social games?
No, it wasn't there from the beginning.
Initially, it was called the "Social Team."
There wasn't a specific team name; it was just a team formed with the goal of creating a social game.
After that, another team was formed and created "Angel Beats!"
However, complaints arose from those on the ground saying, "We can't run this with such a small number of people," so
we decided to do it as one team, and that's how "team Aeca" was formed.
- How many members are there on team Aeca?
There are 9 people
- 9 people!? That's not many..
Yes, it's a small number!
After all, I'm the only one running the server (lol).
the Twitteraccount is also doing it!
— You do that on Twitter too?!
Come to think of it, you reply to all your followers on Twitter, isn't that a lot of work?
Yes, it's tough (laughs).
It's tough, but it's done with the users in mind.
I think it's quite nice to get a reply from the official source.
So I try to reply to messages from users as much as possible, and I've been doing this since the days of "Angel Beats!"
— Visual Arts owns many popular IPs, but
do you plan to continue releasing social games in the future?
That's right. There's the possibility of "Charlotte," which was broadcast as a TV anime in 2015, or perhaps "team Aeca" could launch a new IP
It hasn't been decided yet!
- I'd like to ask you a question about yourself. What made you decide to become an infrastructure engineer?
My previous job was at an independent IT company, and before that I was a freelancer, but..
- Huh? Did you start out as a freelancer?
Yes, that's right. It's rare, isn't it? (laughs)
After working as a freelancer, I started working for an independent IT company, but it wasn't much fun, so I quit. After about a year of going back to being a freelancer, I thought it might be interesting to work at a different company, so I came to this company
When I first joined the company, I was building websites and web systems, and I wasn't doing anything related to infrastructure. I
actually only started working on infrastructure-related things with the previous project, "Angel Beats!".
Since we are not originally a social game company, we do not have any infrastructure engineers or network engineers
Anyway, make sure the server doesn't go down
- What did you pay attention to when developing the server side this time?
The things we paid attention to were load balancing and ensuring the servers wouldn't crash.
Regarding load balancing, we made sure that the system could withstand any issues by distributing the load across different servers.
I couldn't do that with "Angel Beats!"
We started with six cars (laughs)
- That would kill you
I didn't even know how to balance the load
We were just using the usual approach of "this is Web, this is DB, this is memcached," but we got so many users that we
couldn't handle it.
- Did you come here instead of RewriteIM?
RewriteIM is coming
- Even so, with six machines and both the web and database, it's possible that things will die
Yes, you'll die
We started with three web servers, two database servers, and one memcached server, but of course they all crashed, and the web server died first, so we decided we needed to add more web servers, and ended up adding about 30
Because we increased it too much, the number of connections increased too much and we were no longer able to connect to memcached, and eventually the database crashed
However, at that time I hadn't even considered distributing the database, so I set aside a one-month maintenance period and rebuilt everything
- After rebuilding, how many servers did you end up with?
In the end, there were about 80
It was good that we were able to carry out the load test to that extent
- What were some of the difficulties you personally faced this time?
Yamada: It's a load test.
Sashihara: That almost broke my heart
Yamada: I think the database is pretty solid this time..
Sashihara: That's true. But I wonder why it falls
Yamada: That's true. In the end, the problem of crashing under heavy loads hasn't been solved yet
Sashihara: So, it was a pretty close call, or rather, a scary situation when we released it, but
I'm glad that we managed to pull it off.
Yamada: Right now (as of March 2, 2017, when the interview was conducted), we have around 800,000 users, but our previous game, "Angel Beats!", had around 600,000 users after two years of operation, so we've already surpassed that number even though it hasn't even been a month
Sashihara: When we first released it, I was also counting the number of users in the database. The
number of users was increasing at an incredible rate, so I was worried about whether it would be okay.
Yamada: If anything, it was the CDN for downloads that was in a terrible state
Sashihara: The amount of traffic on the CDN was enormous
Yamada: It's about 400TB now
Sashihara: That's about it
Yamada: So I was asking our engineers to somehow reduce the number of downloads, but
of course, there's no way they can do that.
Sashihara: The initial downloads were quite large
Yamada: It's big. It's about 580MB
Sashihara: It goes to the CDN, so it goes that far
Yamada: But even so, the most difficult thing was, to go back to what I was saying, the load testing. It was really tough.
But I'm glad that we were able to do the load testing to that extent.
Sashihara: Thank you for adjusting your shifts to coincide with my night shift
Yamada: Actually, we didn't meet up, we were just at the office (laughs)
Sashihara: When I did it with Buttoushi, I was a little distracted (laughs)
Yamada: We stayed up all night (laughs)
Sashihara: On the weekends before the release..
Yamada: I lived in the company!
Sashihara: I also felt like going home to take a bath (laughs)
Yamada: There's a public bath near my house that opens at 10am, so I always went there
-Finally, please tell us what you thought about using Beyond's services
I feel so grateful that they have been so considerate to me
And the price is cheap!!
The biggest thing is that I was able to sleep without anything happening.
I can sleep, and I don't get calls in the middle of the night, so I no longer get calls around 5:00 AM and
have to work on my home PC, which is a huge relief.
- That's all for the interview. Thank you for today
