Disclaimer: I'm not a serious theorycrafter - if my math/tables/graphs are horribly laughable, sorry! Google Docs' tables were not making my day.
One huge thing that really frustrates me about this game is how number-oriented it is with affixing, yet no one really understands what the numbers do. Like elemental damage... common sense says that monsters weak to elemental damage should take more damage from their weakness, right? What the hell does Ability do? How come my 50% ice sword outdamages my 20% fire rare on an ice-weak enemy?
The big problem with figuring this out is that there is no combat log. Makes things casual-friendly for sure, and the game isn't hard yet by any means, but I know some of us like to go the extra mile on our gear (not me... I can't afford it anymore :<) I wanted to do some concrete testing that would answer MY questions, and and being kind of a scientific person, hopefully my method will convince you of my results.
RA40 - Using 4 3* Vitalauncher+10: 50% Thunder, 50% Ice, 21% Ice, Non-element, no R-ATK/Ability affixes
Mag - lv134 Corvus 89RATK/30ABI, has Buff J but didn't use for testing.
Switched between maxed Ability skill tree and generic RA tree, both with some degree of Standing Snipe (not used)
Used a mix of Shoot/Rappy/Mizer Soul armors to swap ABI/RATK in and out of
lv38 Gilnatches in Area 1 of Free Field Mines. Other than their Thunder weakness, these guys have no weak spots or hard points that I know of, and so Weak Shot Advance should not come into play. They also heal themselves and must have some crazy regen besides, because they're quite hard to kill. I would rush the map until I found a set, and if no element Boost PSE effects happened, it was good to go.
I used launcher basic attacks to record damage, firing at the ground at the mobs' feet. In order to counteract any Standing Snipe/JA buffs, I would jump to shoot, then land before attempting another shot. A range of 50-80 shots were recorded for most sets that looked interesting.
All of my work so far is here, don't go blind:
What I attempted to test
1) Comparison of max 50% elemental weakness damage vs. non-weakness vs. non-elemental (done)
2) Damage comparison of moderate 20% elemental weakness damage vs. non-weakness vs. non-elemental (need to grind up a 21% thunder launcher to match the ice)
3) R-ATK vs. Damage Dealt comparison (done)
4) Ability vs. Min/Max & Average Damage Dealt comparisons (effectively done, but could use a few more numbers near the plateau)
5) What determines the Ability plateau (theoretical)
Some things to think about when deciding ATK vs. Ability:
As I mentioned before, I am using R-ATK vs. a very specific mob using specific weapons. However, Ability and R-ATK both appear to scale linearly (though Ability appears to cap out at its 'plateau'). Since both are linear, there does not seem to be a "sweet spot" as a reason to level both simultaneously.
In fact, it requires 4.4 Ability to increase my average basic attack by 1 damage, and 4.2 R-ATK do to the same, so in my case R-ATK is superior on average.RA does not rely on consistant damage however, so really it appears to be fairly close in value - and everyone hates that low-end Sneak Shot on a Weak Bullet.
When comparing moderate% rares to 50% 1-5* weapons though by looking solely at ATK, remember that the % appears to be a scalable damage increase. It's up in the air though, if the rare weapon's hidden Ability boost increases the average damage enough to make up for it.