Internationalize only when it is absolutely necessary, because it adds a cost to all development. For example, it may not be necessary to put all messages in messages.properties files if the application is simple and has little text.
It isn't just text that needs internationalization. There are different areas of focus that need careful planning:
Language (standard .property files) is the most common solution provided. eg. How do we display the word “order” in Thai?
Measurements. eg. The United States still use inches but the rest of the modern world use millimeters, understandably they don't want to see any measurements in millimeters.
Currencies. Monetary amounts should usually be presented in local currency, and dollar signs that were once hard coded need to be replaced with other symbols like £ or ¥. Some countries need to use amounts in the billions for small purchases, and have local abbreviations for the word 'billion'.
Laws (eg. Tax and auditing rules in different countries and states). Some countries require a display for the amount of GST included in the price.
Cultural differences. For example, some countries don't use family names at all, so having a surname field will add confusion.
Tip: Never internationalize “just in case” you might need it in future. Only ever do it when there is a clear requirement to use the software in non English speaking countries and even then consider if they can get by with a manual English translation sheet. It's expensive to cater for more than one country.
