Slick Forums

Discuss the Slick 2D Library
It is currently Fri Nov 17, 2017 7:17 pm

All times are UTC




Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next
Author Message
PostPosted: Tue Mar 27, 2012 7:57 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
Please tell me the benefits :o Because I can't see them. I could game loads all important ressources on start up (Think up Console Games). If you on an embedded system you probalty want to handle ressource loading differently I give yout hat but on a normal PC? Slick has a nice way to load ressources deferred so providing a loading bar is a 1 hour job.

Attaching new lib to use one or two classes is overkill imho. Is this Open Source? If so I say copy the code if not damn you Apache xD

Oh and could you post the benchmark? Looks kinda strange to me if the difference ist that big and I load all my images in 2 seconds... (around 30 images) ?

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 8:35 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
My game also loads its graphics an resource definitions (drawing offset, animation chains, coloring, attributes, etc.) also on startup. Talking about 91 texture maps (1024x1024) containing a total of around 56000 graphics along with the resource definition tables with around 200000 entries.

If you don't optimize loading there the player gets annoyed over the loading time even if you display a loading bar.

For the benchmark... I'll post it later, don't got access to it now.

But basically I compared this:
Code:
    public static String loadSource(InputStream in) throws SlickException {
        try {
                BufferedReader br = new BufferedReader(new InputStreamReader(in));
                String line = "";
                String txt = "";
                while ((line=br.readLine()) != null)
                        txt += line + "\n";
                br.close();
                return txt.trim();
        } catch (IOException e) {
                throw new SlickException("could not load source file");
        }
    }


with this:

Code:
    public static String loadSource(InputStream in) throws SlickException {
        try {
                return org.apache.commons.io.IOUtils.toString(in).trim();
        } catch (IOException e) {
                throw new SlickException("could not load source file");
        }
    }


I measured only the runtime this two functions using System.nanoTime().

As file to load I used a lorem ipsum file generated here: http://www.loremipsum.de/

As for the new library... I figure you are right. Looking at the rest of Slick we could use like two methods of that whole library so its overkill. I offer to write a utility implementation for loading files in a generalized way once this is looked at (so I can change my repository according to the outcome).

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 8:58 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
Well, I guess that's something davedes could change. I don't know if it's good to use String class with a simple "+". A better approch would be to use a StringBuffer. Also I would like to see the average time of loading since Java will alsways have heavy times on start up.

I Browsed through the code and the don't do anything which StrinBuffer does not. Their just doing the byte reading so I wonder why it's that faster :/ I will test that to.

And I don't think any player get's annoyed. Console Players are like this: "Well the game needs loading it's a console after all". PC Gamer (GAMER not the average asus buying laptop user) will not see much problems in loading. Loading is really something which is only important if you don't have the resources (embedded) and have a lot of textures or sounds.
But I'm not against faster loading :)

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 9:37 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
Okay benchmarked it and it's quite amazing to see the difference:

Tested lorem file with a size of 120.769 bytes (I guess that's a realistic size for image loading (The most of mine are even smaller).

Times ( on 1000 runs per test) :
  • Without Stringbuffer : 55 ms - 58 ms
  • With Stringbuffer : 50 ms - 55 ms (Java does quite the optimization on String concats it seems)
  • With Apache Commons IOUtils: 1 ms

Stunning loading time imho. I still not sure if we should include apache stuff... Do you know if we could just take the class and use it?

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 10:47 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
The problem is not the concating of the strings.

Because
"abc" + "def"

is basically:
buf = new StringBuffer("abc")
buf.append("def")
buf.toString();

This kills the performance if you do the + operation very often but thanks to the hotspot optimizing of Java that is speed up pretty good afterall.

The performance killer is the BufferedReader.readLine() Because this function essential reads the stream byte by byte and looks for "\r", "\n" and "\r\n". Once this is located it stops reading, creates a new string from the byte data and returns it. The much faster method is to create a temporary char array with a fixed size (1024 or 2048) and read the data with a BufferedReader into this array.

Then append the array that was read to the string buffer using StringBuffer.append(char[] data, int offset, int length) and repeat until the stream is fully read. That is imo the fastest way to read data. (Using NIO channels can be done too but that is only significant faster for filesystem resources because you can use FileChannel.transferTo(Buffer) in this case. And this is REALLY fast. But since most of the time the resources are classpath resources that barely makes a difference.

I don't have my Java development environment here. But maybe you can try something like this:
Code:
   public static String loadSource(InputStream in) throws SlickException {
      try {
         final StringBuffer sBuffer = new StringBuffer();
         final BufferedReader br = new BufferedReader(new InputStreamReader(in));
         final char[] buffer = new char[1024];

         int cnt;
         while ((cnt=br.read(buffer, 0, buffer.length)) > -1) {
            sBuffer.append(buffer, 0, cnt);
         }

         return sBuffer.toString().trim();
      } catch (IOException e) {
         throw new SlickException("could not load source file");
      }
   }


Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 4:05 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1482
Updated. There are some other areas (AngelCodeFont, etc) that could also be improved with the same method -- in which case we might want to move it to some Utils type class.

(I didn't include the final trim() -- I don't see why we should silently do that?)

Also, I'm starting work on moving all the ARBShaderObject stuff to SGL in order to utilize GL20 if the system supports it.


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 8:39 pm 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
I send in a pull request that should contain a first version of IO Utilities trimmed down to the needs to Slick.

See if everything is cool and feel free to merge the changes in.

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Tue Mar 27, 2012 11:37 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1482
I looked through Slick -- all the other classes that use file loading in Slick depend on line-breaks (readLine) for parsing. I take back my earlier statement... I now see no benefit in moving the small ShaderProgram convenience method to a standalone IOUtil type class. At least not until there are other classes that would also benefit from it.

Code:
I send in a pull request that should contain a first version of IO Utilities trimmed down to the needs to Slick.

It looks great, but IMO this is a bit overkill and doesn't belong in Slick's codebase.

Just my two cents.


Top
 Profile  
 
PostPosted: Wed Mar 28, 2012 7:03 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
Read operations can be applied to:

  • org.newdawn.slick.font.HieroSettings#HieroSettings(InputStream) - Loading hiero definition file
  • org.newdawn.slick.AngelCodeFont#parseFnt(InputStream) - Loading Angelcode font metrics
  • org.newdawn.slick.PackedSpriteSheet#loadDefinition(String, Color) - Loading sprite sheet definition
  • org.newdawn.slick.opengl.shader.ShaderProgram#readFile(InputStream) - Loading the shader
  • And every application that uses Slick and need to load text data from a file.

That is just what I saw when looking over one time. Its possible that I missed some spots.

Also the utility is not only about handling the file loading properly but also about ensuring the proper handling of InputStreams. Something that was not done in even one of the implementations mentioned before.

The utility I wrote is after all a very small one with limited abilities. It just "look" a lot due the documentation and the many different implementations that use each other. There are like 4 functions in there that actually DO the IO operations. The rest are supporter methods to ease the usage.

Nitram

_________________
http://illarion.org


Last edited by Nitram on Wed Mar 28, 2012 7:17 am, edited 1 time in total.

Top
 Profile  
 
PostPosted: Wed Mar 28, 2012 7:15 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
Maybe the dev wants to use the method too. So Slick has now a nice and fast IOUtil class for super fast loading.
I will look at it too if I get some free time :D

Edit:
Why to you create a pull request btw? Just push the code into the dev branch. That's way easier for us :o


Edit²:

Pweee~ That's really a lot of code there o_O Could you just push it in the util package? I tend to say that we might just want to add the methods we need in the classes and drop this heavy class, since we don't want to create more bloat right?

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Wed Mar 28, 2012 1:55 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1482
Maybe open up a separate thread to discuss it, and to get some more opinions. I personally don't think it is consistent with the rest of Slick's coding style, and would rather see it done in a single class.

Nitram wrote:
org.newdawn.slick.font.HieroSettings#HieroSettings(InputStream) - Loading hiero definition file
org.newdawn.slick.AngelCodeFont#parseFnt(InputStream) - Loading Angelcode font metrics
org.newdawn.slick.PackedSpriteSheet#loadDefinition(String, Color) - Loading sprite sheet definition

All of these would require IOUtils.readLines, which does the same thing as the current implementation but slower; it requires iterating over all lines in a files twice rather than once.

There are a lot of useful classes out there (StringUtils, Bag, etc) that users have the freedom to choose from or write themselves. We shouldn't try to encompass every possible need of a user; ideally we should be trying to reduce clutter and minimize Slick.


Top
 Profile  
 
PostPosted: Wed Mar 28, 2012 2:12 pm 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
R.D. wrote:
Edit:
Why to you create a pull request btw? Just push the code into the dev branch. That's way easier for us :o
I'm not a Slick-Developer. I can't push anything into your repository.

R.D. wrote:
Edit²:
Pweee~ That's really a lot of code there o_O Could you just push it in the util package? I tend to say that we might just want to add the methods we need in the classes and drop this heavy class, since we don't want to create more bloat right?
Its basically just documentation... not that much code at all.

For davedes comments: There are some points there I tend to disagree with. But I will move that to a new topic in order to avoid cluttering this one.
Edit: There :arrow: http://slick.javaunlimited.net/viewtopic.php?f=26&t=4765

Nitram

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Thu Mar 29, 2012 7:18 am 
Offline
Game Developer
User avatar

Joined: Thu Mar 03, 2011 6:22 pm
Posts: 534
@davedes
I think you are right here. I thought that was the reason you implemented a loadSource method. I still don't care much about loading time. Not for 10kb image or anything. IOUtils sure is fast but... I load anything on startup anyway. I even init all my states there (since initiating them before the loading would ref images that don't exist xD)

_________________
Current Projects:
Image Mr. Hat I
Image Vegan Vs. Zombies
Projects:
RadicalFish Engine - Build on top of Slick2D, Ideas, Bugs? Open an Issue ticket!


Top
 Profile  
 
PostPosted: Thu Mar 29, 2012 7:51 am 
Offline
Regular

Joined: Sun Oct 30, 2011 4:47 pm
Posts: 184
Location: Mittweida, Saxony, Germany
Please... along all arguments keep in mind that Slick is not only used for small games. Loading minimal amount of graphics and sounds doesn't take long no matter how you load anything. But Slick is also used for larger scaled projects with > 80MB of images and resources that need to be load.

_________________
http://illarion.org


Top
 Profile  
 
PostPosted: Thu Mar 29, 2012 5:00 pm 
Offline
Slick Zombie

Joined: Sat Jan 27, 2007 7:10 pm
Posts: 1482
R.D. wrote:
I still don't care much about loading time. Not for 10kb image or anything. IOUtils sure is fast but... I load anything on startup anyway.

I agree with Nitram on loading speeds -- we should be as efficient as possible, and try to make Slick usable for "big games."

As for IOUtils, it is no faster than the current implementation in AngelCodeFont/PackedSpriteSheet/HieroSettings -- it uses the same file reading method, and would only exist for a design perspective. In that sense I agree with Nitram -- it makes more sense to have a "single source of responsibility" which is why I originally brought up the idea of IOUtils.

Nitram, my reasoning for your IOUtils being "overkill"...
Methods like "copyLarge" or "closeQuietly" are user-specific and not needed by Slick. Simply because we have an IOUtils class doesn't mean we should try to encompass everything the user may potentially desire; e.g. for the same reason that we don't include a StringUtils class in Slick. Users can easily implement these on their own, and using their own methods (e.g. Java 7 file IO).

I also don't agree with unnecessary method overloading, IMO this falls under "trying to encompass every possible need of the user":
Code:
     public static String toString(final URI input, final String charsetName)
        throws IOException {
        return toString(input.toURL(), charsetName);
    }


I'm not trying to attack you Nitram -- I just don't agree on all points. ;) I would be in favour of a small IOUtils class that is consistent with Slick's "baby Java" coding style and creates a single point of responsibility for file reading.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 63 posts ]  Go to page Previous  1, 2, 3, 4, 5  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group