Discussion Performance per litre - Round 4

[Steven]

I think you are both misunderstanding, this formula isnt sqrt(performance) it's the sqrt(performance^2) ie performance ^1.

As for Mini Builds... I agree there's an issue either way. I think that any build with integrated cpu should be separated anyway. The whole coolnesss of SFF is to keep the modularity of a regular PC but compactify it. But I think skewing the formula to not truly be performance per liter isn't the answer to "protect" from the integrated builds.

 

[hrh_ginsterbusch]

I only read "sqrt" and read "squirt"

Also, you're too windows-centric. Needs a bench suite that is platform-agnostic. I'd keep it at x86 though.

cu, w0lf.

 

[chinevo]

TBH, why are we not using a straight and simple calculation as it was done in PPL round one?

View attachment 3445

we are talking about Performance / Liter and not Performance / Square Liter, Squareroot Performance / Liter, or whatever...

maybe add some other benchmarks or rules for the next round but keeping it simple is often the best solution - IMHO

Performance (GPU) * Performance (CPU) = Square Performance
Maybe (Performance GPU + Performance CPU) / Volume ?

 

[Steven]

Thank you for understanding it was square performance

Adding them (arithmetic mean) leads to imbalance between the scaling of the benchmarks. This is the point of geometric means: sqrt(cpu*gpu). In this simple equation the relative scaling is handled. It's quite simply taking the squared performance and square rooting it

This is actually the simplest equation possible that includes both.

 

[REVOCCASES]

I think the biggest concern of the original formula was, that builds under 1L kinda break it

the easiest solution is to prevent build volume values below <1 in the formula - so if we just use mm^3 instead of cm^3 all should be good and fair for all

my idea:



as example, I calculated some of the entries (where dimensions were given):

@guryhwa Rome Build: PPV = 313.856
@petricor Build: PPV = 230.230
@chinevo EM780 Build: PPV = 110.983
@REVOCCASES "BR1KY" Build: PPV = 100.368
@articnova P3 Tiny Build: PPV = 92.143
@SFFMunkee N0RDR1 Build: PPV = 23.735

if you just look at those builds without any calculation, the performance / size ratio looks more correct to me than previous formulas

anyway, it's just a idea - we can also keep using the original calculation or base it on @Steven 's idea - but no matter what we agree on, we should prevent <1 values

 

[Steven]

Thanks Revo and others for continuing to consider this.

After analyzing it again I don't think <1L breaks any of these formulas, they're all smooth functions in the domain (0,inf]

I see where you're coming from but I think, unfortunately, the 1L delineation doesn't protect from some SoC Minis popping up in the 1-2L space like the MS-A1 or some integrated-on-board builds like the MS-01 or any Laptop even in their default configuration.

Chinevo noticed an interesting situation in the space and capitalized on it brilliantly. Now it's inspired a need to better define the limits moving forward if we want a space for the powerful modular builds.

If you want to protect powerful hobbyist builds (stuffing a 4090 in 5L is an incredible feat) then I think it's best to separate Integrated-on-board and/or SoC builds from the modular component builds.

 

[REVOCCASES]

After analyzing it again I don't think <1L breaks any of these formulas, they're all smooth functions in the domain (0,inf]

well, they don't break the math but the question is, if the math we have been used is still the way to go forward in the next round or if we want to have a more fair competition for everyone...

just like @morj already noted below:

By the way, this whole thing got me thinking about the PPL formula. Why are we measuring performance per cm^4.5?

Explanation: 1 liter is 1000 cm^3 (cubic centimeter). So the formula has Build Volume^1.5, which makes it (cm^3)^1.5 which is (cm^4.5). Performance per something that is over 4 dimensions

Moreover, for builds of, say 4L in volume we have 4^1.5 = 8, so actually 2 times the volume (of penalty) due to this exponent.

On the other hand for a build of 0.25L volume we have 0.25 ^ 1.5 = 0.125, so actually 2 times LESS the volume due to this exponent.

So basically the exponent in the formula IMMENSELY favors the < 1L builds due to the way exponents work. We are looking at like 4x advantage over a "normal" build.

EXAMPLES:


@petricor's build without the exponent: (34,565*33,304)/(5)/100 = 2302305.52

@chinevo's build without the exponent: (12775 * 2688) / ((0.198 + 0.053)) / 100 = 1368095.61753

Is this feature of the PPL formula intentional?

----------

I see where you're coming from but I think, unfortunately, the 1L delineation doesn't protect from some SoC Minis popping up in the 1-2L space like the MS-A1 or some integrated-on-board builds like the MS-01 or any Laptop even in their default configuration.

Chinevo noticed an interesting situation in the space and capitalized on it brilliantly. Now it's inspired a need to better define the limits moving forward if we want a space for the powerful modular builds.

personally I do not want to exclude <=1L projects or mini-PCs from the competition, especially since I'm just 0.3L away from a sub 1L build...

so again, if we define the volume by mm^3 instead of cm^3, the biggest issue is resolved and mini-PC builds can still join the competition without being favored in the rating

 

[morj]

Thanks Revo and others for continuing to consider this.

After analyzing it again I don't think <1L breaks any of these formulas, they're all smooth functions in the domain (0,inf]

I see where you're coming from but I think, unfortunately, the 1L delineation doesn't protect from some SoC Minis popping up in the 1-2L space like the MS-A1 or some integrated-on-board builds like the MS-01 or any Laptop even in their default configuration.

Chinevo noticed an interesting situation in the space and capitalized on it brilliantly. Now it's inspired a need to better define the limits moving forward if we want a space for the powerful modular builds.

If you want to protect powerful hobbyist builds (stuffing a 4090 in 5L is an incredible feat) then I think it's best to separate Integrated-on-board and/or SoC builds from the modular component builds.

Pretty much what @REVOCCASES said above. The formula is non-linear effects are amplified just because the unit is LITERS. Just change it to mm^3 and you will find a much less steep (but in other aspects the same) "performance per volume".

You said "the 1L delineation doesn't protect from some SoC Minis popping up". It's not about delineation, it's about making the whole thing linear. There is nothing about 1L in Revo's approach in mm^3, just a more fair approach to "volume".

 

[Steven]

I think there's some fundamental misunderstanding or miscommunication here.

Whether you use cm^3 or mm^3 or liters does not change either the formula bias nor the linearity. They are interchangeable statements like "6 in one hand, half dozen in the other"

 

[chinevo]

I think there's some fundamental misunderstanding or miscommunication here.

Whether you use cm^3 or mm^3 or liters does not change either the formula bias nor the linearity. They are interchangeable statements like "6 in one hand, half dozen in the other"

I think he meant (Volume in cm)^1.5 instead of (Volume in liters)^1.5 in original formula. In this case my result is 8635, guryhwa's is 6056, much closer to each other. Because Sub-1 numbers to the power of 1.5 have some anomaly

 

[REVOCCASES]

I think there's some fundamental misunderstanding or miscommunication here.

Whether you use cm^3 or mm^3 or liters does not change either the formula bias nor the linearity. They are interchangeable statements like "6 in one hand, half dozen in the other"

@morj already explained the issue with the old formula when the build volume value is <=1 (liter or whatever) - in that case it does make a difference

Moreover, for builds of, say 4L in volume we have 4^1.5 = 8, so actually 2 times the volume (of penalty) due to this exponent.

On the other hand for a build of 0.25L volume we have 0.25 ^ 1.5 = 0.125, so actually 2 times LESS the volume due to this exponent.

but yeah, there was some misunderstanding here - for your (or my) new proposed formula, this does not really matter

 

[Steven]

I think things got confused. The things that do change the ranking is: (1) whether you use volume ^1.5 vs vol^1 or (2) perf*perf (ie perf^2) vs sqrt(perf*perf) (i.e perf)

Chinevo calculated "I think he meant (Volume in cm)^1.5 instead of (Volume in liters)^1.5 in original formula. In this case my result is 8635, guryhwa's is 6056, much closer to each other. Because Sub-1 numbers to the power of 1.5 have some anomaly" This is just a mental illusion.. the ratio 8635/6056 is the same as 2,730,735/1,915,262.

Whether you throw in mm^3 or L into even the Vol^1.5 Equation doesn't affect the ranking nor their ratios (i.e steepness of the ranking). As a sanity test to show it I ran the Vol^1.5 in both L and mm^3

Volume, L L^1.5 Old Formula Using Liter ^1.5 Ratio to next place Volume, mm^3 mm^3^1.5 (4.5) Old Formula Using mm^3 ^1.5 Ratio to next place 0.251 0.1257507495 2,730,735 1.4258 251000 125750749.5 273.0735 1.4258 2.69 4.4119280366 1,915,262 1.8602 2690000 4411928037 191.5262 1.8602 5 11.1803398875 1,029,622 1.3368 5000000 11180339887 102.9622 1.3368 1.45 1.7460312139 770,215 1.5282 1450000 1746031214 77.0215 1.5282 5 11.1803398875 504,016 1.2045 5000000 11180339887 50.4016 1.2045 4.35 9.0726443224 418,447 1.1181 4350000 9072644322 41.8447 1.1181 3.87 7.6131861267 374,245 1.3739 3870000 7613186127 37.4245 1.3739 1.45 1.7460312139 272,396 1.9190 1450000 1746031214 27.2396 1.9190 6.17 15.3259620579 141,943 1.0387 6170000 15325962058 14.1943 1.0387 1.57 1.9672043615 136,655 0.6121 1570000 1967204362 13.6655 0.6121 4.99 11.1468156439 223,244 1.9531 4990000 11146815644 22.3244 1.9531 2.18 3.2187314271 114,301 1.1127 2180000 3218731427 11.4301 1.1127 5.34 12.3399069689 102,726 0.9949 5340000 12339906969 10.2726 0.9949 2.06 2.9566562194 103,251 1.3993 2060000 2956656219 10.3251 1.3993 2.27 3.4200998523 73,787 1.0437 2270000 3420099852 7.3787 1.0437 3.96 7.8803005019 70,697 1.5988 3960000 7880300502 7.0697 1.5988 3.3 5.9947477011 44,219 0.8391 3300000 5994747701 4.4219 0.8391 4.2 8.6074386434 52,695 2.2176 4200000 8607438643 5.2695 2.2176 8.51 24.8252905522 23,763 1.6571 8510000 24825290552 2.3763 1.6571 5.67 13.5012689404 14,340 #DIV/0! 5670000 13501268940 1.4340 #DIV/0!

There's nothing funny going on near or below 1 in any formula (1.5 power or otherwise) that's giving the tiny builds a leg up, other than Chinevo's modded mini build just being actually a lot more space efficient (less cooling needed in that power regime/all chips integrated etc)

 

[chinevo]

This is just a mental illusion.. the ratio 8635/6056 is the same as 2,730,735/1,915,262.

Omg, you are right

 

[REVOCCASES]

thanks for running the numbers again - seems I had a solid brain fart when doing my napkin math

now to move on, we still need to collect some feedback how to go for the next round... new formula, new benchmarks, or ... ?

 

[chinevo]

I think I know the best formula for Round 5

((CineBench R23 score * Time Spy graphics score)/(Build Volume^1.175))/100

In this case, my ultra-small build produces the same result as Petrichor's ultra-powerful build. Both approaches will be equal at the beginning of the round

 

[chinevo]

Another idea:
Geekbench 6 Multi Core * 3DMark Steel Nomad / Volume mm3

Reasons:
1. Simple
2. No 1.5 power problem
3. Latest and cross platform benchmarks
4. My and petricor’s results are almost equal (40-42)

 

[morj]

There's nothing funny going on near or below 1 in any formula (1.5 power or otherwise) that's giving the tiny builds a leg up, other than Chinevo's modded mini build just being actually a lot more space efficient (less cooling needed in that power regime/all chips integrated etc)

You are right. But I still think the 1.5 power is the culprit.

 

[REVOCCASES]

Another idea:
Geekbench 6 Multi Core * 3DMark Steel Nomad / Volume mm3

Reasons:
1. Simple
2. No 1.5 power problem
3. Latest and cross platform benchmarks
4. My and petricor’s results are almost equal (40-42)

Based on the upvotes it seems we have found a good solution for next round.

Shall we still add a sqrt as @Steven proposed earlier, or is this already good and fair enough for everyone?

 

[Steven]

I'd really appreciate the sqrt so we get the performance to the ^1.. But if you can give me a few days I was about to publish some analyses of the possible tests you mentioned so you can see how they behave

 

[valentine]

i'm here to post a submission, but i agree sub 1L~ cases are OP.
(Liters^1.5)+1 might be enough to fix the issue. (then you're never dividing by a fraction).

Proposition:
(R23*TimeSpy / (Liters^1.5)+1 / 100

Examples:
petricor - (33,304*34,565 / (5^1.5)+1 / 100 = 945,090
chinevo - (12,775*2,688 / (0.251^1.5)+1 / 100 = 305,033





that aside, i set out months ago to break the PPL record with petricor's build in mind. of course, many have beaten this since - but i'm still very happy with my achievement.
i posted about this build here, and will post a full build log in coming.. months?.. as time and energy allow

i copied petricors hardware (MOBO, RAM, CPU, etc.) as much as possible, but fit it all into a 3.3L S4T.
(it's a 4080s not a 4090 but pretty close).

due to power and heat constraints, the result is a build that barely squeaks out over his in PPL score!

Case	S4T (Skyreach 4 Tiny) 3D Printed 2mm Thick Cover
PSU	HDPlex 250W GaN
CPU	Ryzen 9 7950x3D
GPU	4080 Super
Cinebench R23 Multi-Core	29,995
TimeSpy Graphics Score	22,358
Round 4 PPL Score (29,995*22,358) / (3.3^1.5) / 100	1,118,692