PPL - Performance Per Liter, do you SFF?

[cleveland]

However, cost is a huge factor. It is also going to change over time which will throw off the results. I can see grouping the current scores into cost brackets or dividing the current scores by the current cost of items... performance per litte per $ (PPL$). The cost should be something like launch price that doesn't change.

This sounds wrong in so many ways...

 

[Gautam]

@Gautam I just expected that the volume would be more important on the SFF Forum than performance. Now we have almost the same as hwbot.org (world overclockers community) has, 7980XE + 1080Ti easily win in any cases. I expected to see in top something like STX MXM, Kaby G, Udoo, Ryzen Embedded, 3.5” mobos, short GPUs, thin Mini ITX and other progressive SFF platforms. Instead of that we see standart money-dependent rating.

That is because we don’t have any submissions from maxed out versions of those setups.

Even a gaming laptop would probably pass 6 mil out of the box. I might be trying that next...

Cost is frustrating, but trying to take it into account will always get messy. Smaller cases and power solutions tend to be expensive, and the best hardware, large or small, always costs the most.

 

[firewolfy]

We tend to know how generally the cost of different CPU/GPU options, so maybe we can look at the rankings as a tool: see how much performance we can get for the money we want to spend?

 

[VegetableStu]

listing parts alone should be enough. at least we have that context in the future ._. (e.g. inflation)

 

[Vlad502]

IMO good performance measure is IPC: Statuscore

Spoiler: i5 6400t (es) 1162 vs 1.87 IPC CPU-Z stress test

Spoiler: i5 6400t (es) 3.33 IPC stress test all

 

[Duality92]

Round 2 will have adjusted Volume modifier to give more points to lower volume, but not too much so that they get a unfair advantage. I consider round 1 to be a success already, so I don't want to change it up too much apart from this slight change.

I want to thank everyone for the input provided too, this makes will make it more fun for round 2.

Formula for round 2 is ((CPU score * GPU Score))/(Volume^1.5)/1000=round 2 points, but both CPU and GPU benchmarks will be changed (but I'm keeping it a secret to which until we get 25 entries!)

 

[thewizzard1]

@Duality92 Why not instead ADD the CPU and GPU score, divided by the volume?

 

[Duality92]

@Duality92 Why not instead ADD the CPU and GPU score, divided by the volume?

I didn't think it would have such an impact by changing this, but modifying my excel to reflect this, it actually makes a lot of sense, I'm actually going to go with this!

 

[Gautam]

I didn't think it would have such an impact by changing this, but modifying my excel to reflect this, it actually makes a lot of sense, I'm actually going to go with this!

Don't do this. See my earlier post about different types of means. A simple average isn't meaningful when it comes to two different benchmarks and a volume measures (all with completely different metrics)

It's much more accurate to use products and roots and adjust weights if needed.

One accurate way of handling this would be to square the volume, instead of having it as-is or ^1.5.

That would effectively be averaging the CPU perf/liter and GPU perf/liter, which is probably what you guys suggesting to do a simple average are trying to achieve. It would still knock my setup out of #1 easily. Doubling the case volume would require doubling BOTH CPU and GPU performance to keep up. A bit extreme, but probably reasonable.

Edit: See this link. Particularly this:

In this case, our geometric mean very much resembles the middle value of our dataset. In fact, it is equivalent to the median.

Note: the geometric mean will not always equal the median, only in cases where there is an exact consistent multiplicative relationship between all numbers (e.g. multiplying each previous number by 3, as we did). Real world datasets rarely contain such exact relationships, but for those that approximate this sort of multiplicative relationship, the geometric mean will give a closer ‘middle number’ than the arithmetic mean.

 

[Duality92]

Don't do this. See my earlier post about different types of means. A simple average isn't meaningful when it comes to two different benchmarks and a volume measures (all with completely different metrics)

It's much more accurate to use products and roots and adjust weights if needed.

One accurate way of handling this would be to square the volume, instead of having it as-is or ^1.5.

That would effectively be averaging the CPU perf/liter and GPU perf/liter, which is probably what you guys suggesting to do a simple average are trying to achieve. It would still knock my setup out of #1 easily. Doubling the case volume would require doubling BOTH CPU and GPU performance to keep up. A bit extreme, but probably reasonable.

I understand what you mean, one point of a benchmark isn't really equal to one point of the other so we can't just add these.

I think I'll stick with my original formula of ((CPU*GPU)/Volume^1.5)/1000, it gives similar but better, IMO, results.

 

[Gautam]

I understand what you mean, one point of a benchmark isn't really equal to one point of the other so we can't just add these.

Yep, and to drive the point home, just imagine a CPU score like mine with some generic low-profile video card, and a GPU score of 100 (or whatever). It'd still get a better score, performance-wise, than an 8700K and a 1080ti, which clearly isn't what we're going for, and would be a very easy system to game in odd ways.

 

[thewizzard1]

@Gautam I like your answer better - The ratio of perf²/vol² makes everything much more 'fair', especially considering we are a SFF-zone
Scores going up exponentially (and not just (cpu+gpu)/vol) makes incremental and generational performance improvements much more noticeable.

@Duality92 Don't edit the numbers now, let's shoot for round 2! You haven't even seen my numbers, yet!

 

[Duality92]

@Gautam I like your answer better - The ratio of perf²/vol² makes everything much more 'fair', especially considering we are a SFF-zone
Scores going up exponentially (and not just (cpu+gpu)/vol) makes incremental and generational performance improvements much more noticeable.

@Duality92 Don't edit the numbers now, let's shoot for round 2! You haven't even seen my numbers, yet!

Oh I'm just changing scoring in my test tab in Excel to see the effect. I'm not changing anything for this round, only the next.

 

[Vlad502]

My opinion, current formula (CPUZMT*Heaven)/Volume is not fair in terms of SFF. In fact it is not performance per liter, it’s performance*performance per liter.

From my point of view (CPUZMT+Heaven)/Volume or similar will be much more fair.

Lets take those two results:
7179 - Gautam - i9 7960X - GTX 1080 Ti - (11537*4916)/7.9 - Post #232
3381 - petricor - i7 8700k - GTX 1080 - (4349*3887)/5.0 - Post #186

11537/7.9=1460 and 4916/7.9=622
4349/5=870 and 3887/5=777
So PPL GPU for second is better. When CPU PPL is 7500/5=1500, both will better, but complete PPL (current formula) will be lower

One more:
2579 - n4ru - i7 8809G - RX Vega M GH - (2930*1426)/1.62 - Post #181

2930/1.62=1809 and 1426/1.62=880
so, to beat this need 9100/5=1820 and 4500/5=900
DeskMini GTX/RX Z370 need 6300/3.5=1800 and 3200/3.5=914

 

[TinyAudio]

Intel Skull Canyon:
Chassis + PSU = 0.73litres
CPUZ = 1890
Heaven = 317
((1890*317)/0.73)/1000 = 820

820 - TinyAudio - i7 6770HQ - HD580 - ((1890*317)/0.73)/1000 - Post 275
[edited volume]

https://valid.x86.fr/ktfn2a

When your keyboard is 8X the volume of your pc....

After spending the last 8 years on a Mac it has been fun doing these benchmarks. Even building my own PC in the past it was always for silence.

 

[Duality92]

Intel Skull Canyon:
Chassis + PSU = 0.73litres
CPUZ = 1890
Heaven = 317
((1890*317)/0.73)/1000 = 820

820 - TinyAudio - i7 6770HQ - HD580 - ((1890*317)/0.73)/1000 - Post 275
[edited volume]

https://valid.x86.fr/ktfn2a

After spending the last 8 years on a Mac it has been fun doing these benchmarks. Even building my own PC in the past it was always for silence.

Need a picture of your rig my friend

 

[TinyAudio]

Need a picture of your rig my friend

Updated the post with a picture.

 

[InfernoZeus]

Updated the post with a picture.

Sweet keyboard...

 

[Duality92]

Updated the post with a picture.

Added

I'm gonna go add my 8600K rig just to get us closer to 25

 

[Duality92]

https://valid.x86.fr/6ud34z

Love this low leakage binned chip lol, I can run it at 5.2 GHz with 1.42v (put 1.5v for these benches, just for the sake of brute forcing it, because I run it stock regardless usually) on a Noctua L9i It also almost reaches @confusis i7 8700

1046 - Duality92 - i5 8600K - GTX 1070 - (3669.6*2970)/10.4 - Post #280