If your design is right, you don't have to worry about how long it takes to do a String.indexOf(...) call.
If your design is wrong, it won't matter how many nanoseconds you have shaved off a body of code that only loops twenty times – your software will still run at a crawl.
Of course, this does not mean do silly things like use an ArrayList for a cache instead of a HashMap, but if your programmers need to do profiling before they have finished the code, start to worry. The reality is, Java will optimize plenty of code for you behind the scenes, (especially static Strings), and unless you really do something stupid, most of the time you will spend waiting for I/O to get back to you. ie. “Operating System, please give me this file”.