Stop Microsoft
Operating Systems => Linux and UNIX => Topic started by: toadlife on 1 July 2005, 11:47
-
http://toadlife.kicks-ass.net/bsdvswindows/
I'm trying to compare 3D performance of FreeBSD to Windows. So far I've done a set of Quake III benchmarks that show some interesting results. I plan on tossing my spare 40GB ahrd drive into my machine and installing some flavor of linux to see how it does too.
I would also have some doom III benchmarks too, but you can't do benchmarks with the demo version. I hate the game, so I'm not going to buy it just to benchmark it.
Unreal Tournament 2004 also supports benchmarking, but like Doom III, you have to have the full version. I hate UT2004 too, so I'm not going to buy it either.
I think I'm going to have to bust out my Bittorrent to help me get these benchmarks done.
Anyone know of any other 3d games that run on Linux/Windows, and can be benchmarked?
-
well i don't know any games that can run on both of them but i know lots of win games, but UT2004 is Direct 3D not OpenGL, my copy of UT2004 doesn't even support OpenGL , i have to chosse between Software and D3D rendering
-
Try Neverwinter Nights..
I like that game (still!)
-
but UT2004 is Direct 3D not OpenGL, my copy of UT2004 doesn't even support OpenGL , i have to chosse between Software and D3D rendering
The linux version of UT 2004 doesn't support OpenGL?
-
Waitamoment. So, windows beats freebsd hands down, except at maximum resolution. And this is after windows version has been given disadvantage by disabling all GL extensions that would've otherwise been there.
So, what the heck are you benchmarking? How can you say freebsd redeems itself at the end? What if you ordered those framerates in inverse order, you'd see that windows framerates keep steadily improving while freebsd's framerates don't give a fuck?
Also, are you sure that freebsd's rates aren't capped to screen refresh rate or something? That would explain the around-75 values. Why did you disable GL extensions in windows? Why not just add extra tests with them on? Would it be unfair to compare the systems as they actually perform under real world situation?
Sounds like windows is constantly winning even in your biased comparison, and you're calling this "interesting"? Honestly, it was to be expected.
-
maybe the bsd was limited by drivers or summin. windows one shud be fsater, with all the extra secret api's in windows, but really, 75fps is quite acceptable.
-
The linux version of UT 2004 doesn't support OpenGL?
Yes, it does. The Windows version does not. That's what he's looking at.
-
Muzzy - he disabled GL extensions in both versions. Read the top paragraph more carefully.
-
There's something not right with those results. The FPS are almost identical in BSD at nearly every resolution; usually that suggests there's some kind of processing bottleneck. I'm suspecting the Linux emulation is to blame.
If you want to do this fairly you need to find two OpenGL apps that run natively in both operating systems.
-
FreeBSD has no Linux "emulation" - it's translation, much like VMware. I'm going with muzzy's suggestion of screen refresh issues.
-
Muzzy - he disabled GL extensions in both versions. Read the top paragraph more carefully.
That's my point exactly. "Because it didn't work in freebsd, it was only fair to turn it off in windows". WTF? "Because performance in freebsd sucked, it was only fair to turn off all performance improving features in windows as well". Is this what it's saying? WHAT THE HECK IS BEING BENCHMARKED IN HERE, REALLY? Can you answer that? What real world thing does this benchmark measure? If it tests the driver implementations, why does one have to be handicapped when other one fails? Does this benchmark efficiency of library calls? The compiler that was used to compile the software for the specific platform? Anything at all?
To me, this "benchmark" makes no sense whatsoever, and the results aren't very interesting. It only displays what we all knew: windows is superior for this stuff, even if you have to artificially bring it down to same playing field with "competing" operating systems.
-
That's my point exactly. "Because it didn't work in freebsd, it was only fair to turn it off in windows". WTF? "Because performance in freebsd sucked, it was only fair to turn off all performance improving features in windows as well".
The GL extensions in BSD simply didn't work. They weren't slow. If something doesn't work, what's the purpose of using it in a benchmark? Are you saying that leaving them turned on in Windows would make this *more* fair somehow?
This benchmark is simply BSD vs. Windows in Quake 3, but he had to turn something off to make it work properly. Why is that such a mystery? Either way, the bigger problem is the sameness in the BSD framerates across all resolutions. Something else is wrong, and we still don't know who wins this battle, GL extensions or not.
-
So, we want to get some practical results about game performance on two different systems, and because performance features don't work on one platform, they have to be turned off on others to make comparison fair?
Do I have to bend an explanation from an iron wire so you'll get it, too? Heck, let's make opengl comparison ... I'll write software implementation, but I'll only implement ONE FUNCTION! Thus, we'll only benchmark that. As a result, we'll notice that my opengl implementation beats every other implementation in performance! Woo Yay! How about that? Flawed benchmark? So, how much do I have to implement before it stops being flawed? Oh, just basic functionality but anything that'd actually make a difference won't be used?
I can't see what your "fair" benchmark will tell. What do the values mean? Really, tell me, what significance do those numbers have? Of what practical or even theoretical use are they to anyone or anything?
Are we going to compare system bogomips next?
-
So, we want to get some practical results about game performance on two different systems, and because performance features don't work on one platform, they have to be turned off on others to make comparison fair?
Yeah, essentially. It's not FreeBSD's fault that the GL extensions don't work - they don't make them. Quake 3 even uses the Linux OpenGL AFAIK, so that also hurts. If there was a real FreeBSD version of Quake 3, I'd think GL extensions would work there. But there isn't, so they won't.
Are we going to compare system bogomips next?
Sure, I have 5622.98. What's yours? :p
-
Maybe Heavy Gear 2?
And you can get it cheap at ebay.
-
right now in BeOS world
Rudolf Cornelissen is making a 3d driver and he sais :
BTW: I now compared timedemo1 speeds in Linux (nVidia closed source
driver) and BeOS. The Linux driver is 5 times faster than mine (I get
400fps for Q2 in for instance 800x600x32 mode on GF4Ti, MX about 250fps
If I remember correctly)
As I know for sure I am using the HW command available to the max it
becomes clear that they use other commands, or have extra features in
the card that enourmously speeds up the command I use. You'd better
prepare that we will not get those speeds on BeOS for nvidia.
I will publish the measurements and these thoughts on my weblog soon
BTW. It seems like a good idea to at some point do a 3D driver for a
'opensource hardware' 3D card. Even if these cards perform (much) worse
than nvidia's cards in windows, having full docs on such a card would
effectively make it much faster than a nvidia card in BeOs....
so that is interesting. see my website here:
http://beq2.beworld.info/hardware_accel_on_beos.html
-
You are jumping the gun a bit muzzy. My benchmark page is FAR from complete, and I never said that FreeBSD is better than Windows for 3d performance. As I said, I plan on benchmarking more games, and I also plan on doing more Quake benchmarks with various AA/AF settings turned on.
As for the speculation about refreshrate limiting framerates in FreeBSD, they were not. I immediately suspected this myself, and made sure the vertical sync was turned off. Also, when I changed the lighting settings from "Lightmap" to "Vertex", the framerate in FreeBSD jumped to 96fps at 640x480. My monitor runs at 65-85mhz depending on the resolution.
-
There's something not right with those results. The FPS are almost identical in BSD at nearly every resolution; usually that suggests there's some kind of processing bottleneck. I'm suspecting the Linux emulation is to blame.
I think you are right. FreeBSD's linux ABI support is extremely efficient, but I suspect there are certain features of the linux kernel that either can't or don't get emulated properly. I seem to remember similar behavior in America's Army when running it in FreeBSD - changing to lower resolutions would not help framerates very much. I would have to go back and verify that though.
If you want to do this fairly you need to find two OpenGL apps that run natively in both operating systems.
There are none that fit that criteria.
-
sigh...any shit running through any sort of layer and not on the hardware is going to be slower........
not to mention freebsd is more of a server operating system then a desktop one. linux on the other hand.....ive run quake 3 on both, its more stable and faster under linux....then agian, any layers that quake 3 has to go through under linux(like X11 and the kernel) probably has had gcc3 grind on the opimization flags for days whereas windows must run on almost any shit86 boxen. not to mention im betting dev on the linux/freebsd nvidia drivers as well as ati drivers is treated like a second class citizen.
-
ahhhhhh, i know of a native opengl app...its anciet but its native to both platforms...glquake and quake2.
-
I've posted some more benchmarks of Quake III. I am going to try and do a comparison of America's Army. I think I can benchmark it by just loading the maps and viewing the fps. This won't be as good as doing a timedemo, but it's one of the only *modern* games available on both Windows and linux.
-
well i don't know any games that can run on both of them but i know lots of win games, but UT2004 is Direct 3D not OpenGL, my copy of UT2004 doesn't even support OpenGL , i have to chosse between Software and D3D rendering
You probably need to get drivers for your video card, I don't even play games and I know this. But it is a rough estimation that I am over fifty thousand times more intelligent than you. It amazes me how you manage to live in something that small.
-
So, we want to get some practical results about game performance on two different systems, and because performance features don't work on one platform, they have to be turned off on others to make comparison fair?
Do I have to bend an explanation from an iron wire so you'll get it, too? Heck, let's make opengl comparison ... I'll write software implementation, but I'll only implement ONE FUNCTION! Thus, we'll only benchmark that. As a result, we'll notice that my opengl implementation beats every other implementation in performance! Woo Yay! How about that? Flawed benchmark? So, how much do I have to implement before it stops being flawed? Oh, just basic functionality but anything that'd actually make a difference won't be used?
I can't see what your "fair" benchmark will tell. What do the values mean? Really, tell me, what significance do those numbers have? Of what practical or even theoretical use are they to anyone or anything?
Are we going to compare system bogomips next?
Why don't we all get our cameras and post our dongs.
-
Interesting. FreeBSD runs America's army faster than Windows - at *ALL* resolutions. I'll post the benchmarks soon.
-
I think you are right. FreeBSD's linux ABI support is extremely efficient, but I suspect there are certain features of the linux kernel that either can't or don't get emulated properly. I seem to remember similar behavior in America's Army when running it in FreeBSD - changing to lower resolutions would not help framerates very much. I would have to go back and verify that though.
There are none that fit that criteria.
There is an interesting fact that I will point out. The time I will point it out is now. The Linux Kernel has a major proformance option called "premptive kernel". I have no idea exactly what this means however I am well aware of its physical effects on the system. Basically: without it Linux runs as slow as Steve Blammer's metabolism. I have no clue in the entire wet Universe if FreeBSD does whatever this (preemptive kernel) does. Maybe look into that, my dear toadlife.
-
There is an interesting fact that I will point out. The time I will point it out is now. The Linux Kernel has a major proformance option called "premptive kernel". I have no idea exactly what this means however I am well aware of its physical effects on the system. Basically: without it Linux runs as slow as Steve Blammer's metabolism. I have no clue in the entire wet Universe if FreeBSD does whatever this (preemptive kernel) does. Maybe look into that, my dear toadlife.
I have no idea if what you describe is supported in FreeBSD.
Here is a quote from the FreeBSD handbook on linux compatibility..
In a nutshell, the compatibility allows FreeBSD users to run about 90% of all Linux applications without modification. This includes applications such as Star Office, the Linux version of Netscape, Adobe Acrobat, RealPlayer 5 and 7, VMWare, Oracle, WordPerfect, Doom, Quake, and more. It is also reported that in some situations, Linux binaries perform better on FreeBSD than they do under Linux. There are, however, some Linux-specific operating system features that are not supported under FreeBSD. Linux binaries will not work on FreeBSD if they overly use the Linux /proc filesystem (which is different from FreeBSD's /proc filesystem), or i386-specific calls, such as enabling virtual 8086 mode.
Unlike withg Quake III, in FreeBSD the framerates in America's Army increased at the resolution went down all the way down to 640x480, and FreeBSD consitently outperformed Windows in almost every map. There were a few exceptions where Windows beat FreeBSD.
I don't have time to post the becnhmarks right now. I'm leaving for the coast for fourth of july tommorow. I might post them in tne next few days though, if I can ssh into my box.
-
America's Army benchmarks are up.
Quite suprising they were.
http://toadlife.kicks-ass.net/bsdvswindows/#aastart (http://toadlife.kicks-ass.net/bsdvswindows/#aastart)
-
You do realise I am not on your local network and therfore that link is very useless?
I hope so. God I hope so.
-
You do realise I am not on your local network and therfore that link is very useless?
I hope so. God I hope so.
I certainly hope you are not on my local network.
I did a copy/paste job without thinking..
link fixed.
-
Of coarse BSD will have better frames... SP2 has many un-needed services running. This is common sense and you did not need to bench.
try tweaking and modifying xp until it uses the same amount of ram as BSD then bench. It's like comparing a race between 2 cars, but one car has a bunch of shit that weighs it down inside of it.
if you were going to use stock settings, there was no point of benching, FreeBSD would win obviusly(sp).
-
Of coarse BSD will have better frames... SP2 has many un-needed services running. This is common sense and you did not need to bench.
Windows was not running any other programs, except for Geoshell, which is a lightweight explorer replacement. There was no AV or Anti-Spyware running and the firewall was turned off. The difference in idle memory used by Windows/BSD was actually very little. I would say about 87MB for BSD and 125MB in use for Windows. Ina system with 512MB ram, that's nothing.
try tweaking and modifying xp until it uses the same amount of ram as BSD then bench. It's like comparing a race between 2 cars, but one car has a bunch of shit that weighs it down inside of it.
Modern operating systems page out ram for programs that are not doing anything and allocate physical memory for programs that are active. If there were any shortage of memory there would have been hard disk activity when the games were running. In both OS's there was no hard disk activity aside from the initial loading of the maps.
if you were going to use stock settings, there was no point of benching, FreeBSD would win obviusly(sp).
How would it obviously win? It sure as hell didn't obviously win with Quake III.
-
It sure as hell didn't obviously win with Quake III.
the only reason it didn't was because of GL extentions. :thumbdwn:
-
What results were you looking at? Even with GL Ext disabled, WIndows beat out FreeBSD in all resolutions except for 1600x1200, and even at 1600x1200 the difference is tiny.
I can go around disabling all the services in the world in Windows, but it's not going to yield 20 extra frames per second.
I've give it a try later on. I'll disable a bunch of services and redo some of the benchmarks to see if there is any improvement. I seriously doubt if there will be any.
-
Okay, while scarfing down PB&J's on my lunch break I loaded Windows up and disabled the following services in Windows:
3dm
automatic Updates
computer browser
distributed link
error reporting service
help and support
ipsec services
print spooler
remote registry
secondary logon
security center
server
tcppi netbios helpoer
task scheduler
themes
web client
windows firewall
warsvr
windows time
wireless zero configuration
workstation
This brought down the memory usage in Windows to 85MB. That's two less MB than FreeBSD without X loaded. I loaded AAO up and did a couple of tests:
Mountian Pass no AA/no AF
1600x1200 New results: 48FPS Old Results: 48fps
640x480 New results: 59FPS Old Results: 57Fps
Mountain pass 4xaa / 4x af
1600x1200 New Results: 15FPS Old Results: 14FPS
The difference is miniscule.
Now I have to go back and turn all my services back on. :mad: