r/deeplearning • u/CivilSnow5625 • 13h ago
Automatic Model-Agnostic Compression Algorithm
I made [Sigularty](https://github.com/DewanshShah/Sigularty), an algorithm that combines multiple compression techniques, takes in any model and chooses which techniques to use, and also finds hyperparameters for a few of the techniques. This is one of my first proper projects, and I would love feedback. It is completely free and open source, and these are a few results the algorithm achieved by itself, without any human fine-tuning
|Model|Acc Drop|Compression Ratio|CQI|Speedup|Size (MB)|Accuracy (%)|Latency (ms)|
|:-|:-|:-|:-|:-|:-|:-|:-|
|`Resnet50`|**2.20%**|**8.00x**|**9.897**|**1.28x**|90.66 -> 11.34|66.60 -> 64.40|6.977 -> 5.451|
|`vit_b_16`|**0.00%**|**4.51×**|**4.526**|**1.00x**|327.59 -> 72.67|77.00 -> 77.00|14.96 -> 14.90|
|`efficientnet_b0`|**2.80%**|**2.61x**|**2.284**|**0.90x**|15.95 -> 6.11|91.80 -> 89.00|8.238 -> 9.124|
|`bert_base`|**4.20%**|**6.61x**|**6.449**|**1.02x**|417.66 -> 67.17|86.20 -> 82.00|10.393 -> 10.209|
|`distilbert`|**5.60%**|**9.55x**|**7.049**|**0.78x**|255.42 -> 26.75|82.40 -> 76.80|5.441 -> 6.957|
|`distilgpt2`|**0.00%**|**2.00x**|**2.903**|**1.45x**|312.48 -> 156.24|87.20 -> 87.20|8.559 -> 5.896|
1
u/wealthyintercession 13h ago
That vit_b_16 result is wild, zero accuracy drop with a 4.5x size cut is the kind of thing that makes me side-eye the eval setup a bit