AWS Database Blog

How to determine whether Kaigen (Japan era name transition) affects your MySQL compatible engines running on RDS

If you have software or systems that support Japanese customers, and if your software or systems need to display a Japan era name, you may need to make changes to display the new name. The new Japan era name comes into effect on May 1, 2019, when the current Japanese Emperor abdicates.

In this blog post, I provide background information on Kaigen, the Japan era name transition. Then I explain in detail the investigation methods that I used to see whether MySQL compatible engines running on RDS will be affected by the Japan era name transition. I am happy to report that they are not affected by the transition!

Introduction to Kaigen

Japan has an original calendar schema called Gengō (“元号”) in Japanese.

When a current Emperor abdicates and a new Emperor is enthroned, the Japan era name transitions to a new era name, and the first year of the new era begins. This Japan era name transition is called Kaigen (“改元”), which means “Change the Gengō” in Japanese.

The Japan era name is still commonly used in Japan, especially for government systems and official papers. Some software supports the Japan era name. For example, Windows stores Japan era names in the registry, Oracle databases have a calendar file that renders Japan era names, and the strftime() function in the glibc library has functionality that renders Japan era names correctly.

On April 30, 2019, the current Emperor of Japan will abdicate, so the Japan era name will transition to the new era name beginning May 1, 2019. Therefore, software that supports Japan era names needs to be updated to the new era name.

The new Japan era name is scheduled to be announced on April 1, 2019. That leaves only a very short period, less than a month, to update software with the new name.

Understanding the conditions

The only Linux built-in software that has a capability of rendering the Japan era name is the strftime() function in glibc.

strftime() is a function that formats date and time, and it is able to render a date with the Japan era name. If you specify a format that includes the %EY conversion specification in the Japanese (ja_JP) locale, the Japan era name displays correctly on your console. The list of Japan era names is stored in a locale file, typically installed in /usr/share/i18n/locales/ja_JP by the glibc package.

For example, the date command uses strftime() to format date and time. When using the date command, if you specify %EY with the Japanese locale, it renders the Japan era name. The Japan era names are usually stored with EUC-JP character encoding, so I piped the output to nkf to encode to UTF-8:

$ LANG=ja_JP date +%EY -d 2019-01-01|nkf
平成31年

How do you know if your software will be affected by the transition, and therefore needs to be updated? It needs to be updated if it meets these conditions:

  • It uses strftime() imported from glibc, and accepts the %EY conversion specification
  • It supports the ja_JP locale
  • It is capable of rendering the Japan era name

Investigation #1: Using strftime() imported from glibc and accepting %EY conversion specification

When using strftime(), it’s easy to check by grepping the source code. For example, you can clone MySQL 5.6.43 source code into your local machine from GitHub and search strftime() usage in the source code, as shown in the following example:

[~/work/]$ git clone https://github.com/mysql/mysql-server.git -b mysql-5.6.43 mysql-server-5.6.43
[~/work/]$ cd mysql-server-5.6.43
[~/work/mysql-server-5.6.43]$ grep -r strftime *
cmake/info_macros.cmake.in:    "use warnings; use POSIX qw(strftime); "
cmake/info_macros.cmake.in:    "print strftime \"%F %T %z\", localtime;")
libevent/http.c:		if (strftime(date, sizeof(date),
scripts/mysqld_multi.sh:use POSIX qw(strftime getcwd);
scripts/mysqld_multi.sh:    print strftime "%a %b %e %H:%M:%S %Y", localtime;
scripts/mysqld_multi.sh:    print LOGFILE strftime "%a %b %e %H:%M:%S %Y", localtime if ($date_flag);
storage/ndb/test/crund/tws/tws_cpp/Driver.cpp:        const int nchars = strftime(dest, size, format, localtime(&now));
storage/ndb/test/crund/src/crundndb/Driver.cpp:        const int nchars = strftime(dest, size, format, localtime(&now));
storage/ndb/test/src/getarg.c:    strftime(timestr, sizeof(timestr), "%B %e, %Y", localtime(&t));

Based on the output, we can see that strftime() is not used in MySQL server runtime, and almost all usages are with hard-coded format specification, without %EY. The usage in libevent/http.c is also with hard-coded format specification, without %EY:

if (strftime(date, sizeof(date),
                          "%a, %d %b %Y %H:%M:%S GMT", cur_p) != 0) {
                          evhttp_add_header(headers, "Date", date);
                  }

My investigations proved that MySQL compatible engines running on RDS don’t use strftime() with either customer-configurable format specifications or with hard-coded format specification with %EY. The following data shows the results of my investigation of each engine:

MySQL 5.6

  • Server components using strftime() are:
    • libevent (used in InnoDB memcache plugin)
    • mysqld_multi … Not used in RDS
    • NDB test suite … Not used in RDS
  • libevent uses strftime() with hard-coded format specification without %EY

MySQL 5.7

  • Server components using strftime() are:
    • libevent (used in InnoDB memcache plugin)
    • mysqld_multi … Not used in RDS
    • NDB components (test suite, MySQL Cluster Configurator) … Not used in RDS
  • libevent uses strftime() with hard-coded format specification without %EY

MySQL 8.0

  • Server components using strftime() are:
    • sql/log_event.cc
    • mysqld_multi … Not used in RDS
    • NDB components (test suite, MySQL Cluster Configurator) … Not used in RDS
    • MySQL Router (and Duktape library) … Not used in RDS
  • sql/log_event.cc uses strftime() with hard-coded format specification without %EY

MariaDB 10.0

  • Server components using strftime() are:
    • mysqld_multi … Not used in RDS
    • Storage engines which are unsupported in RDS (Spider, Mroonga, CONNECT and NDB)

MariaDB 10.1 – 10.4

  • Server components using strftime() are:
    • mysqld_multi … Not used in RDS
    • Storage engines which are unsupported in RDS (Spider, Mroonga, CONNECT and NDB)
  • Utility tools which are not used in RDS
    • Innotop
    • innobackupex

Aurora MySQL 5.6/5.7

  • Server components using strftime() are:
    • Some internal loggings
    • Some external libraries imported for Aurora specific functionalities
    • mysqld_multi … Not used in RDS
    • NDB components (test suite, MySQL Cluster Configurator) … Not used in RDS
  • All components use strftime() with hard-coded format specification without %EY

Investigation #2: Supporting the ja_JP locale

In RDS, the system locale is fixed to en_US.UTF-8. However, lc_time_names, a parameter that controls the locale of temporal strings, is modifiable in RDS MySQL/MariaDB. For Aurora MySQL 5.6/5.7, it is modifiable in the DB cluster parameter group. You can check whether an engine version supports lc_time_names by using the following AWS CLI command:

$ aws rds describe-db-parameters --query 'Parameters[?ParameterName==`lc_time_names`].[ParameterName,AllowedValues,IsModifiable]' --db-parameter-group-name default.mysql5.6
[
    [
        "lc_time_names",
        "ar_AE,ar_BH,ar_DZ,ar_EG,ar_IN,ar_IQ,ar_JO,ar_KW,ar_LB,ar_LY,ar_MA,ar_OM,ar_QA,ar_SA,ar_SD,ar_SY,ar_TN,ar_YE,be_BY,bg_BG,ca_ES,cs_CZ,da_DK,de_AT,de_BE,de_CH,de_DE,de_LU,el_GR,en_AU,en_CA,en_GB,en_IN,en_NZ,en_PH,en_US,en_ZA,en_ZW,es_AR,es_BO,es_CL,es_CO,es_CR,es_DO,es_EC,es_ES,es_GT,es_HN,es_MX,es_NI,es_PA,es_PE,es_PR,es_PY,es_SV,es_US,es_UY,es_VE,et_EE,eu_ES,fi_FI,fo_FO,fr_BE,fr_CA,fr_CH,fr_FR,fr_LU,gl_ES,gu_IN,he_IL,hi_IN,hr_HR,hu_HU,id_ID,is_IS,it_CH,it_IT,ja_JP,ko_KR,lt_LT,lv_LV,mk_MK,mn_MN,ms_MY,nb_NO,nl_BE,nl_NL,no_NO,pl_PL,pt_BR,pt_PT,ro_RO,ru_RU,ru_UA,sk_SK,sl_SI,sq_AL,sr_RS,sv_FI,sv_SE,ta_IN,te_IN,th_TH,tr_TR,uk_UA,ur_PK,vi_VN,zh_CN,zh_HK,zh_TW",
        true
    ]
]

$ aws rds describe-db-parameters --query 'Parameters[?ParameterName==`lc_time_names`]'  --db-parameter-group-name default.aurora5.6
[]

$ aws rds describe-db-cluster-parameters --query 'Parameters[?ParameterName==`lc_time_names`]'  --db-cluster-parameter-group-name default.aurora5.6
[]

This means that if MySQL/MariaDB engine versions have a capability to render the Japan era name in their date formatting function, and if the function is aware of lc_time_names, it could be affected by the Japan era name transition.

Investigation #3: Individually having a capability to render the Japan era name

The only function to change the date format in MySQL is DATE_FORMAT().

DATE_FORMAT() is implemented by MySQL from scratch without using strftime(), as shown in the following examples:

In all of the engine versions, the list of supported format characters (in the switch/case clause) does not have %EY or any other format string that can render the Japan era name.

As we found in investigation #2, customers can change the locale for time format to ja_JP, but DATE_FORMAT() doesn’t have the capability of rendering the Japan era name in the first place, so the Japan era name transition won’t cause any problem, even if lc_time_names is modifiable.

Conclusion

The Japan era name transition won’t affect any MySQL compatible engines running on RDS because:

  • strftime() is not used in server components, either with customer configurable format specifications or with hard-coded format specification with %EY
  • DATE_FORMAT() also does not use strftime() and does not have a capability to render the Japan era name

Therefore, if you are using a MySQL compatible engine running on RDS, you don’t need to take any action to prepare for the Japan era name transition.

About the Author

Yoshihiko Matsuzaki is a database engineer with the Relational Database Services (RDS) team at Amazon Web Services.