Slick Forums

Please tell me the benefits Because I can't see them. I could game loads all important ressources on start up (Think up Console Games). If you on an embedded system you probalty want to handle ressource loading differently I give yout hat but on a normal PC? Slick has a nice way to load ressources deferred so providing a loading bar is a 1 hour job.

Attaching new lib to use one or two classes is overkill imho. Is this Open Source? If so I say copy the code if not damn you Apache xD

Oh and could you post the benchmark? Looks kinda strange to me if the difference ist that big and I load all my images in 2 seconds... (around 30 images) ?

_________________
Current Projects:
Mr. Hat I
Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!

My game also loads its graphics an resource definitions (drawing offset, animation chains, coloring, attributes, etc.) also on startup. Talking about 91 texture maps (1024x1024) containing a total of around 56000 graphics along with the resource definition tables with around 200000 entries.

If you don't optimize loading there the player gets annoyed over the loading time even if you display a loading bar.

For the benchmark... I'll post it later, don't got access to it now.

But basically I compared this:


with this:



I measured only the runtime this two functions using System.nanoTime().

As file to load I used a lorem ipsum file generated here: http://www.loremipsum.de/

As for the new library... I figure you are right. Looking at the rest of Slick we could use like two methods of that whole library so its overkill. I offer to write a utility implementation for loading files in a generalized way once this is looked at (so I can change my repository according to the outcome).

Nitram

_________________
http://illarion.org

Well, I guess that's something davedes could change. I don't know if it's good to use String class with a simple "+". A better approch would be to use a StringBuffer. Also I would like to see the average time of loading since Java will alsways have heavy times on start up.

I Browsed through the code and the don't do anything which StrinBuffer does not. Their just doing the byte reading so I wonder why it's that faster :/ I will test that to.

And I don't think any player get's annoyed. Console Players are like this: "Well the game needs loading it's a console after all". PC Gamer (GAMER not the average asus buying laptop user) will not see much problems in loading. Loading is really something which is only important if you don't have the resources (embedded) and have a lot of textures or sounds.
But I'm not against faster loading

_________________
Current Projects:
Mr. Hat I
Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!

Okay benchmarked it and it's quite amazing to see the difference:

Tested lorem file with a size of 120.769 bytes (I guess that's a realistic size for image loading (The most of mine are even smaller).

Times ( on 1000 runs per test) :

• Without Stringbuffer : 55 ms - 58 ms
• With Stringbuffer : 50 ms - 55 ms (Java does quite the optimization on String concats it seems)
• With Apache Commons IOUtils: 1 ms

Stunning loading time imho. I still not sure if we should include apache stuff... Do you know if we could just take the class and use it?

_________________
Current Projects:
Mr. Hat I
Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!

The problem is not the concating of the strings.

Because
"abc" + "def"

is basically:
buf = new StringBuffer("abc")
buf.append("def")
buf.toString();

This kills the performance if you do the + operation very often but thanks to the hotspot optimizing of Java that is speed up pretty good afterall.

The performance killer is the BufferedReader.readLine() Because this function essential reads the stream byte by byte and looks for "\r", "\n" and "\r\n". Once this is located it stops reading, creates a new string from the byte data and returns it. The much faster method is to create a temporary char array with a fixed size (1024 or 2048) and read the data with a BufferedReader into this array.

Then append the array that was read to the string buffer using StringBuffer.append(char[] data, int offset, int length) and repeat until the stream is fully read. That is imo the fastest way to read data. (Using NIO channels can be done too but that is only significant faster for filesystem resources because you can use FileChannel.transferTo(Buffer) in this case. And this is REALLY fast. But since most of the time the resources are classpath resources that barely makes a difference.

I don't have my Java development environment here. But maybe you can try something like this:


Nitram

_________________
http://illarion.org

Updated. There are some other areas (AngelCodeFont, etc) that could also be improved with the same method -- in which case we might want to move it to some Utils type class.

(I didn't include the final trim() -- I don't see why we should silently do that?)

Also, I'm starting work on moving all the ARBShaderObject stuff to SGL in order to utilize GL20 if the system supports it.

I send in a pull request that should contain a first version of IO Utilities trimmed down to the needs to Slick.

See if everything is cool and feel free to merge the changes in.

Nitram

_________________
http://illarion.org

I looked through Slick -- all the other classes that use file loading in Slick depend on line-breaks (readLine) for parsing. I take back my earlier statement... I now see no benefit in moving the small ShaderProgram convenience method to a standalone IOUtil type class. At least not until there are other classes that would also benefit from it.


It looks great, but IMO this is a bit overkill and doesn't belong in Slick's codebase.

Just my two cents.

Read operations can be applied to:

• org.newdawn.slick.font.HieroSettings#HieroSettings(InputStream) - Loading hiero definition file
• org.newdawn.slick.AngelCodeFont#parseFnt(InputStream) - Loading Angelcode font metrics
• org.newdawn.slick.PackedSpriteSheet#loadDefinition(String, Color) - Loading sprite sheet definition
• org.newdawn.slick.opengl.shader.ShaderProgram#readFile(InputStream) - Loading the shader
• And every application that uses Slick and need to load text data from a file.

That is just what I saw when looking over one time. Its possible that I missed some spots.

Also the utility is not only about handling the file loading properly but also about ensuring the proper handling of InputStreams. Something that was not done in even one of the implementations mentioned before.

The utility I wrote is after all a very small one with limited abilities. It just "look" a lot due the documentation and the many different implementations that use each other. There are like 4 functions in there that actually DO the IO operations. The rest are supporter methods to ease the usage.

Nitram

_________________
http://illarion.org

Maybe the dev wants to use the method too. So Slick has now a nice and fast IOUtil class for super fast loading.
I will look at it too if I get some free time

Edit:
Why to you create a pull request btw? Just push the code into the dev branch. That's way easier for us


Edit²:
Pweee~ That's really a lot of code there o_O Could you just push it in the util package? I tend to say that we might just want to add the methods we need in the classes and drop this heavy class, since we don't want to create more bloat right?

_________________
Current Projects:
Mr. Hat I
Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!

Maybe open up a separate thread to discuss it, and to get some more opinions. I personally don't think it is consistent with the rest of Slick's coding style, and would rather see it done in a single class.


All of these would require IOUtils.readLines, which does the same thing as the current implementation but slower; it requires iterating over all lines in a files twice rather than once.

There are a lot of useful classes out there (StringUtils, Bag, etc) that users have the freedom to choose from or write themselves. We shouldn't try to encompass every possible need of a user; ideally we should be trying to reduce clutter and minimize Slick.

I'm not a Slick-Developer. I can't push anything into your repository.

Its basically just documentation... not that much code at all.

For davedes comments: There are some points there I tend to disagree with. But I will move that to a new topic in order to avoid cluttering this one.
Edit: There http://slick.javaunlimited.net/viewtopic.php?f=26&t=4765

Nitram

_________________
http://illarion.org

@davedes
I think you are right here. I thought that was the reason you implemented a loadSource method. I still don't care much about loading time. Not for 10kb image or anything. IOUtils sure is fast but... I load anything on startup anyway. I even init all my states there (since initiating them before the loading would ref images that don't exist xD)

_________________
Current Projects:
Mr. Hat I
Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!

Please... along all arguments keep in mind that Slick is not only used for small games. Loading minimal amount of graphics and sounds doesn't take long no matter how you load anything. But Slick is also used for larger scaled projects with > 80MB of images and resources that need to be load.

_________________
http://illarion.org

I agree with Nitram on loading speeds -- we should be as efficient as possible, and try to make Slick usable for "big games."

As for IOUtils, it is no faster than the current implementation in AngelCodeFont/PackedSpriteSheet/HieroSettings -- it uses the same file reading method, and would only exist for a design perspective. In that sense I agree with Nitram -- it makes more sense to have a "single source of responsibility" which is why I originally brought up the idea of IOUtils.

Nitram, my reasoning for your IOUtils being "overkill"...
Methods like "copyLarge" or "closeQuietly" are user-specific and not needed by Slick. Simply because we have an IOUtils class doesn't mean we should try to encompass everything the user may potentially desire; e.g. for the same reason that we don't include a StringUtils class in Slick. Users can easily implement these on their own, and using their own methods (e.g. Java 7 file IO).

I also don't agree with unnecessary method overloading, IMO this falls under "trying to encompass every possible need of the user":


I'm not trying to attack you Nitram -- I just don't agree on all points. I would be in favour of a small IOUtils class that is consistent with Slick's "baby Java" coding style and creates a single point of responsibility for file reading.