[WIP] ArzBaller - the CLI for packing dbrs

Stormcaller · August 16, 2016, 2:28am

oh my ~~

jiaco · August 16, 2016, 4:38am

I am pretty sure this is not yet 100% functional, but thanks to Stormcaller, it is a lot closer. Stormcaller, if you can rerun the test you did yesterday, that would be great. As far as I can tell, I handled the issues and the arz are now very close in size to the originals. I tested Crucible quickly, but need to take a cat to the Vet and am out of time for now.

jiaco · August 16, 2016, 6:12am

Wait for v0.2 I found another bug, will update asap.

jiaco · August 16, 2016, 6:25am

Alright, so now I have extracted and recompiled the SurvivalMode.arz and things seem to work, but that is not to say that closer examination will not find other bugs.

Version 0.2 is up in Source:

And the binary test folder has the arz I used in testing:

Command line usage from within the extracted test folder:

arzBaller.exe SurvivalMode.arz sm
arzBaller.exe sm mSurvivalMode.arz

backup your SurivialMode.arz and install mSurvivalMode.arz as SurvivalMode.arz in the correct location.

there are no resources available or used by this process, if I find out anyone is using this tool to provide people with unpaid DLC, this whole process will just get aborted.

also, for those that do not get the reference, on unix, you can use tar and gzip to make an archive format call tar.gz or tar-ball. that is where arzBaller get’s it name, I did not let Depravities name it, if that is what you were thinking.

Stormcaller · August 16, 2016, 6:28am

EDIT: THIS WAS FOR V0.1

I extracted most sensible combinations if other people want to help with crates tools and comparing diffs themselves:

(numbers in dir names refer to filenames in .arz files)
https://a.uguu.se/lrAOonuXZB2D_catdogcatdog.7z (static-1.out not included, along with batteries)
I also sent a pm for a segfault case.

I took the original arz(unmodified, file named SurvivalModeORIG.arz), played crucible with it to confirm it works, it did.

I then extracted the SurvivalModeORIG.arz) with arzballer(dir: 00-survival-original-extracted-via-baller), then packed it with arzballer again(file: SurvivalMode-packed-from-folder-00.arz)
Side effects:

top right ui gone
numbers above mastery bar are gone, skill leveling is impossible

file size was different, but only by a few kbs. I think this is because you always remove one value from an array(Race005;Race006 is Race005Race006 on your arz)

(right side is your tool, left is external)
on comparing unpacking of your tool and others,

this is always true:

bios:

what was missing previously is now 0:

first values missing:

minus values being off by 1 seems fixed:

I think the array values missing a delimeter or values could be a solid bug, and others are minor less reliable results, so check that one out.

jiaco · August 16, 2016, 6:48am

Ugh, sorry I did not think you grabbed v0.1, but yes, it was still bugged for array field values. I have tested v0.2 for this and I think it is fixed now.

Stormcaller · August 16, 2016, 6:56am

I played an arz with v0.2, now, crucible works, except for the top-right ui and the timer things. My characters don’t bug anymore.

rewards work etc.

so, how about the rest, should I diff again? any idea about the top right missing?

if the top right thing can be fixed, enter the… CrucibleLT, crucible with xp gain

jiaco · August 16, 2016, 7:15am

Nice, thanks Stormcaller, about to test a repacked main database right now, will look into this top-left issue next.

Stormcaller · August 16, 2016, 7:25am

aight, let me know if there comes something I can help with.

jiaco · August 16, 2016, 7:31am

Well, there are issues, will have to take care of this when time presents, which is not just yet.

main database (repack) will not load and there must be some records in UI for Crucible that I am unclear about and borking.

I see, might be multiple issues, Crucible has new templates, so I will integrate that. As to what is up with a complete database.arz, that will be much harder to diagnose.

Stormcaller · August 16, 2016, 8:14am

I extracted the main db with your tool, tried repackaging it again with your tool. After 10 minutes, I give up. It just stalls, I think. (edit: for comparison, compiling crucible db takes mere seconds)

I’m in it for the crucible, anyways hopefully the templates are enough to fix it

jiaco · August 16, 2016, 8:26am

The main game database takes its sweet time but finishes for me.

I have to deal with RL atm, but this is on my todo list…I need to remake the BSO for fieldType information with all these new fields from Crucible.


MISSING FIELD TYPE INFORMATION FOR forceCollision from records/creatures/defense
s/banner_defense.dbr
MISSING FIELD TYPE INFORMATION FOR marketScrollTable from records/creatures/npcs
/merchants/merchanttbl_s01.dbr
MISSING FIELD TYPE INFORMATION FOR offensiveSleepModifier from records/game/bala
ncingadjustment_survivalmode_enemies01.dbr
MISSING FIELD TYPE INFORMATION FOR eventBeginSound from records/game/gameengine.
dbr
MISSING FIELD TYPE INFORMATION FOR eventEndSound from records/game/gameengine.db
r
MISSING FIELD TYPE INFORMATION FOR eventFailSound from records/game/gameengine.d
br
MISSING FIELD TYPE INFORMATION FOR eventNpcSound from records/game/gameengine.db
r
MISSING FIELD TYPE INFORMATION FOR eventNpcSound2 from records/game/gameengine.d
br
MISSING FIELD TYPE INFORMATION FOR mutatorFx from records/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR mutatorSound from records/game/gameengine.dbr

MISSING FIELD TYPE INFORMATION FOR normalPrefixRareSuffixModifierCommonChest fro
m records/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR rareBothPrefixSuffixModifierCommonChest from
records/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR rarePrefixNormalSuffixModifierCommonChest fro
m records/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR rarePrefixOnlyModifierCommonChest from record
s/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR rareSuffixOnlyModifierCommonChest from record
s/game/gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR survivalAdjustmentElite from records/game/gam
eengine.dbr
MISSING FIELD TYPE INFORMATION FOR survivalAdjustmentNormal from records/game/ga
meengine.dbr
MISSING FIELD TYPE INFORMATION FOR survivalAdjustmentUltimate from records/game/
gameengine.dbr
MISSING FIELD TYPE INFORMATION FOR treasureRoomSound from records/game/gameengin
e.dbr
MISSING FIELD TYPE INFORMATION FOR waveEndSound from records/game/gameengine.dbr

MISSING FIELD TYPE INFORMATION FOR affectsPlayers from records/game/mutators/mut
atorpak_monster_01.dbr
MISSING FIELD TYPE INFORMATION FOR eliteMutator from records/game/mutators/mutat
orpak_monster_01.dbr
MISSING FIELD TYPE INFORMATION FOR mutatorName from records/game/mutators/mutato
rpak_monster_01.dbr
MISSING FIELD TYPE INFORMATION FOR normalMutator from records/game/mutators/muta
torpak_monster_01.dbr
MISSING FIELD TYPE INFORMATION FOR ultimateMutator from records/game/mutators/mu
tatorpak_monster_01.dbr
MISSING FIELD TYPE INFORMATION FOR bossBonus from records/game/playerscore.dbr
MISSING FIELD TYPE INFORMATION FOR championBonus from records/game/playerscore.d
br
MISSING FIELD TYPE INFORMATION FOR commonBonus from records/game/playerscore.dbr

MISSING FIELD TYPE INFORMATION FOR heroBonus from records/game/playerscore.dbr
MISSING FIELD TYPE INFORMATION FOR questBonus from records/game/playerscore.dbr
MISSING FIELD TYPE INFORMATION FOR survivalScoreEquation from records/game/playe
rscore.dbr
MISSING FIELD TYPE INFORMATION FOR loot4Name4 from records/items/lootchests/ches
tloottables/chestloot_g01.dbr
MISSING FIELD TYPE INFORMATION FOR loot2Name6 from records/items/lootchests/ches
tloottables/chestloot_h01.dbr
MISSING FIELD TYPE INFORMATION FOR maxPlayerLevel10 from records/proxies/poolsba
sic/chthonianvoid_ambush_t1.dbr
MISSING FIELD TYPE INFORMATION FOR maxPlayerLevel11 from records/proxies/poolsba
sic/chthonianvoid_ambush_t1.dbr
MISSING FIELD TYPE INFORMATION FOR minPlayerLevel15 from records/proxies/poolsba
sic/zombies_t1.dbr
MISSING FIELD TYPE INFORMATION FOR maxPlayerLevelChampion8 from records/proxies/
poolshero/chthoniandreadguard_hero.dbr
MISSING FIELD TYPE INFORMATION FOR maxPlayerLevelChampion13 from records/proxies
/poolshero/dranghoul_hero.dbr
MISSING FIELD TYPE INFORMATION FOR poolEpic2 from records/proxies/tier01waves/pr
oxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR poolEpic3 from records/proxies/tier01waves/pr
oxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR poolLegendary2 from records/proxies/tier01wav
es/proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR poolLegendary3 from records/proxies/tier01wav
es/proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR weightEpic2 from records/proxies/tier01waves/
proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR weightEpic3 from records/proxies/tier01waves/
proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR weightLegendary2 from records/proxies/tier01w
aves/proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR weightLegendary3 from records/proxies/tier01w
aves/proxy_w04_p03a.dbr
MISSING FIELD TYPE INFORMATION FOR poolEpic4 from records/proxies/tier04waves/pr
oxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR poolEpic5 from records/proxies/tier04waves/pr
oxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR poolEpic6 from records/proxies/tier04waves/pr
oxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR poolLegendary4 from records/proxies/tier04wav
es/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR poolLegendary5 from records/proxies/tier04wav
es/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR poolLegendary6 from records/proxies/tier04wav
es/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightEpic4 from records/proxies/tier04waves/
proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightEpic5 from records/proxies/tier04waves/
proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightEpic6 from records/proxies/tier04waves/
proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightLegendary4 from records/proxies/tier04w
aves/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightLegendary5 from records/proxies/tier04w
aves/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR weightLegendary6 from records/proxies/tier04w
aves/proxy_w05_p02a.dbr
MISSING FIELD TYPE INFORMATION FOR soundName10 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName11 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName12 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName13 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName14 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName15 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName16 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName17 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName18 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName19 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName20 from records/sounds/npcdialogue/s
pak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName5 from records/sounds/npcdialogue/sp
ak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName6 from records/sounds/npcdialogue/sp
ak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName7 from records/sounds/npcdialogue/sp
ak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName8 from records/sounds/npcdialogue/sp
ak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName9 from records/sounds/npcdialogue/sp
ak_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight16 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight17 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight18 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight19 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight20 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight21 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight22 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight23 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight24 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight25 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight26 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight27 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight28 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight29 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR weight30 from records/sounds/npcdialogue/spak
_master_compliments.dbr
MISSING FIELD TYPE INFORMATION FOR soundName21 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName22 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName23 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName24 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName25 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName26 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName27 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName28 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName29 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR soundName30 from records/sounds/npcdialogue/s
pak_master_taunts.dbr
MISSING FIELD TYPE INFORMATION FOR survivalBackground from records/ui/survivalpa
ne/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier1 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier10 from records/ui/survival
pane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier2 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier3 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier4 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier5 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier6 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier7 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier8 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplier9 from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplierGlow from records/ui/surviv
alpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplierPulse from records/ui/survi
valpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalMultiplierRollover from records/ui/su
rvivalpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalScore from records/ui/survivalpane/su
rvivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalScoreRollover from records/ui/surviva
lpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalSubtractorGlow from records/ui/surviv
alpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTimer from records/ui/survivalpane/su
rvivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTimerGlow from records/ui/survivalpan
e/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTimerRollover from records/ui/surviva
lpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTributeGlow from records/ui/survivalp
ane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTributeRollover from records/ui/survi
valpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalTributes from records/ui/survivalpane
/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalWaveTier from records/ui/survivalpane
/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR survivalWaveTierRollover from records/ui/surv
ivalpane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR timerFailedSound from records/ui/survivalpane
/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR timerTenSecondsSound from records/ui/survival
pane/survivalpane_table.dbr
MISSING FIELD TYPE INFORMATION FOR tributeEarnedSound from records/ui/survivalpa
ne/survivalpane_table.dbr
COMPILE sm2 to output arz: smode2.arz
C:\Users\jiaco\Documents	estArzBaller>

jiaco · August 16, 2016, 10:51am

Here is a version 0.3 for you Stormcaller

You should be able to get your mod done with this. Looking forward to it.

I made a bit of a mess in the source, will finish making this ready for when GD has >1 DLC and upload the source again when the changes are all in place. For now, this was a bit of a hack to just get Stormcaller moving forward with his mod.

Thanks again for your help with this.

WareBare · August 16, 2016, 11:40am

yeah this is getting really really close now.

I extracted the grimmest you had in the first version and than packed it back just to compare the size

with the initial version the real one was about 635kb and after arzBaller did its thing it was over 900kb

now after arzBaller is done its at 638kb, this is so close

EDIT:
when it comes to my mod - nothing works npcs and items disappeard.

jiaco · August 16, 2016, 12:06pm

I think Stormcaller and I have figured out one of the problems that might explain everything, if we are lucky. Quite a bit of re-coding to do but simply I cannot just store fieldName -> fieldType as different templates can use the same fieldName for different types of data.

Some fields have floats others have equations/strings but since I mapped that fieldName to float, the equations strings are getting lost as 0.000000

system · August 16, 2016, 12:23pm

offtop

2stormcaller
I don’t make public mods.
Any good one requires amount of time i don’t have. And eventually ordinary modding gets boring.
Also anyone who have purpose to release mod for crucible gotta be careful, just in case. People with original data still can join to such game.

Stormcaller · August 16, 2016, 2:24pm

well since you won’t make it, can you tell us how did you manage to mod the crucible? I can’t get the recompiled arc to work.

FWIW, I am not after big changes. I just want to fix exp/devotion stuff. More of a “tweak”

jiaco · August 16, 2016, 3:17pm

I am stopping for tonight, but should be able to finish this whole template-specific fieldName->fieldType resource tomorrow…lets hope this is the “main” problem and we can start using this tool for modding then.

jiaco · August 17, 2016, 2:31am

Axioms for Programmers:

Do not write code when you do not want to write code
Always close any files you open - undefined behavior is a bitch

Anyway, wasted at least 4 hours finding the bug I inadvertently introduced trying to add full template support. I blame the claustrophibic confines of this stupid IDE and a single monitor, but whatever, bug is gone, code seems to run, the arzs are still not the exact same size, but I have now extracted and re-packed even the entire game database.arz and can get into game with it.

Call it version 400 or 0.4, here is source:

and the Binary:

Stormcaller · August 17, 2016, 2:56am

testing, see you soon. may take some time, gotta earn tributes
edit:
diff, this time

edit2: i sold my everythign to get the last 10k iron. i hope you are happy >_>