r/DataHoarder • u/CARLOSKARBA1 • 7h ago
r/DataHoarder • u/TheBBP • Jun 14 '26
Mod Update Mod update: regarding the flood of HDD/SSD price posts, and AI slop.
Regarding HDD price posts:
We empathise with your plight and hope HDD / SSD / RAM prices will come down soon.
However, we don’t need screenshots of high HDD prices to be posted in the sub multiple times a day, everyone already knows that the prices are high.
As such, from this point on the mod team will be removing all posts about high prices, with exception for posts made on Free-Post-Friday’s, or posts which actually have new and meaningful information or discussion.
Regarding AI content and AI projects:
As always AI written posts & comments are not allowed on this subreddit, please report any Ai generated content you see.
The mods have recently been cracking down on the flood of AI generated projects, notably ones of low quality that are nothing new.
If someone has the skill to use GitHub, they’d almost certainly have the skill to ask an AI to code yet another YT-DLP / FFMPEG wrapper themselves.
However, there are useful tools that have been made from AI generated code. If they are something truly useful or new, we do allow them with prior approval.
TL;DR:
- Mods will be removing [high HDD price] posts, except for meaningful discussion or Fridays.
- Posting AI generated projects/tools needs mod approval beforehand, and a link to the GitHub repository.
- Please keep reporting ai-slop.
r/DataHoarder • u/Big-Hamster-4116 • 7h ago
Backup Old anime Dvds (2000s)
Hello I'm guy and I got some old Dvds from my anime club and I wanted to know if it would make sense to save them? There are like 45-60 dvds.
r/DataHoarder • u/FallDeeperAlice5268 • 5h ago
Discussion Just purchased a 16TB drive.. is this how it starts? How did you start out?
Basically what the title says. Is this how it starts lol?
I've got a small server type thing arriving tomorrow and I bought this drive off ebay and it claims to be new but we'll see. I'll buy the second when I can afford to lol.
How did you guys start out?
r/DataHoarder • u/redditunderground1 • 8h ago
Discussion What is the excuse for Blu-ray / M-Disc burners skyrocketing in price? AI doesn't use them.
I have 6 Blu-ray burners. (4 LG and 2 ASUS) I thought I'd add another couple as backups. The LG has gone from $87 to $565. The ASUS from $125 to $389. What is the excuse for Blu-ray / M-Disc burners skyrocketing in price? AI doesn't use them.
r/DataHoarder • u/TheyTheirsThem • 2h ago
Question/Advice Six Excess Passports
I have been a 3-deep person for a long time. With the higher drive prices, I did an inventory and found 1TB, 2TB, 3TB, (2) 4TB, and one 5TB passport as excess drives where I increased storage incrementally over the years. They started as primary backups, and then went layer 3 when I bought a larger one, and most have been replaced by 5, 8, and 16TB in the chain. They have around 150 starts and 500-700 hours meaning powered up 2x a week for about 3-5 hrs for about 1.5 yrs before being sidelined for a newer bigger drive. So not new, but not overly used? What % of new BestBuy price would be a good price to offer over FB marketplace or Craigslist. I would demo drive at pickup. Local chain store had 2TB for $210 and 4TB for $310, hence my taking stock, literally. =8O
PS Is 102 TB too much? ;-) My friend thinks so.
r/DataHoarder • u/VORGundam • 1d ago
Free-Post Friday! HDD score! Haggled them down from $20/GB to $15/GB
r/DataHoarder • u/OakTreesForBurnZones • 23h ago
Discussion What has happened to this sub? Wiki and other sidebar items are now "Forbidden"
I used the wiki to find some great sources of data for a class last March. Is there somewhere else where these torrents can be found? /u/archivist ?
r/DataHoarder • u/Hopeful-Truth-1672 • 16h ago
Hoarder-Setups Stormforge N10 - LED mod
So, after receiving my Stormforge N10 with a few status LEDs knocked clean off the board in shipping, and getting a partial refund from Aliexpress, I decided I'd change the colour scheme to pink (for drive presence) and purple (for activity).
Somewhat inspired by a Hello Kitty themed audio mixer/effects rack build, I decided to go with another sanrio character, Kuromi. Because all my homeys stan the goth rabbit.
The factory backplane in the N10 runs the activity LEDs directly off the P11, the drive activity pin on the SAS/SATA power connector.
The original blue LEDs draw about 1.7mA each. I've seen conflicting reports as to how much current this pin is supposed to sink, from 100uA to 15mA. The most reliable resource I could find is a manual for Seagate SAS drives that said 3.3V @ 15mA. The factory LEDs run from the 5V rail but *should* in theory have enough voltage drop across them to stay below 3.3V.
Purple LEDs are not very efficient. I'm running them at about 14mA, which *should* be within limits if most drives are similar to seagate's, but given how expensive hard drives are right now I don't really want to risk it.
So I build this little buffer circuit to run the activity LEDs. It draws about 220uA @ 0.6V on the activity pin, and should be able to handle >40mA for LEDs
r/DataHoarder • u/FewSimple9 • 1d ago
Free-Post Friday! 5+ PB Destroyed
This is based on a post by u/SubstantialWing9238. i didn’t want to hijack their post, so I figured I’d share it here instead.
Something like this, plus another cart, is pretty common with large corporations. When you have this many drives, it’s often easier to just shred everything than pay someone to wipe them, sort through them for resale, and see if any data can be recovered.
This batch was around 300 drives, and with a specialized shredding truck, the whole thing takes about 45 minutes to a hour.
r/DataHoarder • u/Agitated-Meaning-314 • 23h ago
Question/Advice How Many HDD Hours and Power On Counts Too Many?
Facebook marketplace in my area is selling four 8 TB hard drives each with 60,000 power on hours for $140 each. The drives also have power on counts and the high 200s. How high is too high when considering a used drive? I would likely keep 16 TB in Cold storage but I don't want to waste money if the drives are too used.
All of the research I see shows people referring to the power on counts more than the hours, but no one gives their thoughts on what the range of good versus too high is. Thoughts are appreciated.
r/DataHoarder • u/thetoastmonster • 1d ago
Question/Advice legacygames is shutting down, anyone got a tool to download all installers in your account?
https://legacygames.com/legacy-games-closure-faq-everything-you-need-to-know/
Legacy Games is shutting down, encouraging everyone to download their owned games before the shutdown date as once it has passed it will no longer be possible to do so.
I saw someone mention that the installer is temporarily placed in your temp folder when you choose to install.
Has anyone got a tool for grabbing all installers for all owned games in your account?
r/DataHoarder • u/CodOk4768 • 10h ago
Question/Advice Is it possible for the community to restore HatchXR?
I’ve made a r/lostmedia post about HatchXR, but I said that these remain: “The only stuff that is remaining is archives from Wayback Machine (unfortunately no playable games :(), Socials, and a iOS app. The iOS app is now useless because there is no actual projects to play (there used to)”. With no playable games, how much can be restored?
r/DataHoarder • u/Better_Chapter4643 • 6h ago
Question/Advice Any Telegram Client to bypass restrictions?
Is there any Telegram client like the ‘aka’, which allows you to get round the restrictions?
Ever since aka stopped working, I’ve been searching.
r/DataHoarder • u/SubstantialWing9238 • 2d ago
Discussion Last week was 19.2TB, this week RIP 154TB
51 x 3TB and 2 x 500GB. I would love to keep them or sell them but it’s a big company and all they care about is getting the data destruction certificate. They are pretty old though and definitely have tens of thousand of hours on them.
r/DataHoarder • u/MikeTheMic81 • 16h ago
Hoarder-Setups Compressing/Transcoding in an AI-Pocolypse
I'm not sure if I'm allowed to post scripts in here, but I wanted to give a breakdown of how I just free'd up a massive amount of space on my arrays! If it's not allowed, let me know and I'll take it down. It's been a super long process. I posted my scripts opensource so hopefully this is allowed. It's been a few months process in my case, and I WISH I had thought about this back when I was running a few HDD's worth of content, not when I had almost a PB of data. lol
My Current Hardware for those who want to test (The Python Scripting requires NVidia GPU's Though)
Windows 11
CPU: 10900k (Doesn't matter for any of these steps because it's using GPU power)
GPU: EVGA FTW3 RTX 3090
Ram: 64GB DDR4 3600 (Also not important for any of these steps)
Here's a breakdown of how I freed up nearly 80% of my media space, and 30% of my NVMe space (used for game installs) without actually deleting anything. This may not help everyone, but I don't think I've seen such a comprehensive solution. For many, this is going to be a little back-end work first and then automated going forward. It's also great for anyone at the start of their hoarding.
--- Media ---
The biggest problem I seemed to have is H.264. For most TV-Shows and Movies, H.265 is going to be good enough (and in the case of older TV-Show maybe even better than the original) Here's the steps I took:
Radarr and Sonarr:
1) Create a new custom format named x265 / HEVC with the regular expression \b([xy]265|hevc)\b (leave everything else unchecked)
2) Go into "Profiles" and, in my case, I had all my media going into "Any" in that profile there's 2 things to change, for "Group Order" prioritize HDTV-1080p as the best of the best by dragging it to the top of the list and saving it. Set "Upgrade Until" to HDTV-1080p. Under "Custom Formats" make sure that x265 / HEVC is there and give it a score of +1000. Then save the profile.
What this does is prioritizes your library to love H.265 files. It can't get enough of them! Once this is saved, you'll be able to go to "Cutoff Unmet" and see all the files that have been using up all your storage. Any new media you download from this point on will automatically look for H.265 releases first and only default to H.264 if it can't find an H.265 release. For radarr, you may want to avoid Remux as they tend to be giant files even in H.265 and just uncheck anything that says "Remux".
For anyone with a huge library, or awful GPU's and CPU's on the server but fast download speeds, this is going to be a lifesaver for files. Now, clear through your cutoff unmet (Longest process but beats what is happening next)
Older files you can't find online anymore easily:
There's going to be older shows that just DON'T have easy to find H.265 copies. I can think of a certain show my wife likes that came out in Pre-2000 and the seasons can hit over 80-120GB each!
1) Install FFmpeg from Powershell (or Terminal admin in Windows 11)
winget install Gyan.FFmpeg
2) Install Python from Powershell (or Terminal admin in Windows 11)
winget install Python.Python.3.14
3) Ensure they're both installed with:
ffmpeg -version
python --version
4) Once we are all installed, this is the Python script I created. It's going to work in 2 ways:
- It will ask you which drive you want to convert
- It will ask you if you want to convert a specific folder on that drive
- If you say "Y" put in the full path to the folder
- If you say "N" it will ask how many you want to convert (Some huge libraries can take days depending on how many files it needs to convert. Starting off small might be a good idea)
It will then look for h.264 in the specified folder, it will automatically convert it to h.265 10-Bit with heavy compression. It took me AGES to tune it in just right. You may want to avoid doing this on anime or cartoons, but especially older media, which is most likely what you couldn't find already in h.265, this will make it look BETTER than the original h.264. It will also create a log file on anything it transcodes and show you the old file sizes as well as the new file sizes.
import os
import subprocess
import json
import sys
import re
from datetime import datetime
# ==================== CONFIGURATION ====================
# Supported video extensions to check
VIDEO_EXTENSIONS = ('.mp4', '.mkv', '.avi', '.mov', '.m4v', '.flv', '.ts')
# Directory where the logs will be saved
LOG_DIRECTORY = r"C:\TaskSchedulerTasks\Transcode Logs"
# =======================================================
def get_video_stream_info(file_path):
"""Uses ffprobe to extract codec, resolution, and pixel format safely."""
cmd = [
'ffprobe',
'-v', 'error',
'-select_streams', 'v:0',
'-show_entries', 'stream=codec_name,pix_fmt,width,height',
'-of', 'json',
file_path
]
try:
result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True)
data = json.loads(result.stdout)
if 'streams' in data and len(data['streams']) > 0:
return data['streams'][0] # Return the dictionary directly
except Exception:
pass
return None
def generate_arr_compliant_filename(file_path):
"""Removes old resolution/codec tags and forces '1080p.HEVC.10-Bit'."""
directory, filename = os.path.split(file_path)
base, ext = os.path.splitext(filename)
# Regex to strip out old quality/codec metadata
pattern = re.compile(r'[-._\s]?(?:h[-._]?264|x264|avc|2160p|4k|1080p|720p)[-._\s]?', re.IGNORECASE)
cleaned_base = pattern.sub('.', base)
cleaned_base = re.sub(r'\.+', '.', cleaned_base).strip('.- ')
new_filename = f"{cleaned_base}.1080p.HEVC.10-Bit{ext}"
temp_output_path = os.path.join(directory, f"TEMP_TRANSCODING_{new_filename}")
final_output_path = os.path.join(directory, new_filename)
return temp_output_path, final_output_path
def convert_to_h265_10bit(input_path, output_path, scale_to_1080p=False):
"""
Executes FFmpeg utilizing NVENC hardware acceleration on the RTX 3090.
Enforces a strict bitrate ceiling to smash file sizes down to the 610MB benchmark.
"""
cmd = ['ffmpeg']
if scale_to_1080p:
cmd.extend([
'-hwaccel', 'cuda',
'-c:v', 'h264_cuvid',
'-resize', '1920x1080',
'-i', input_path
])
else:
cmd.extend([
'-hwaccel', 'cuda',
'-i', input_path
])
cmd.extend([
'-c:v', 'hevc_nvenc', # Use NVIDIA RTX Hardware encoder circuit
'-pix_fmt', 'p010le', # Force 10-Bit color profile
'-preset', 'p7', # Max quality NVENC encoding path
'-cq', '28', # Deep efficiency quality target
'-maxrate', '2200k', # Strict ceiling to compress down to 600MB-700MB bracket
'-bufsize', '4400k', # Rate control tracking buffer
'-c:a', 'copy', # Pass audio tracks through untouched
'-c:s', 'copy', # Pass subtitles through untouched
output_path
])
try:
print(f"🎬 Starting High-Compression Transcode: {os.path.basename(input_path)}")
subprocess.run(cmd, check=True)
return True
except subprocess.CalledProcessError:
print(f"❌ FFmpeg conversion failed for {input_path}")
return False
def format_size(bytes_size):
"""Formats file sizes into human-readable versions."""
for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
if bytes_size < 1024.0:
return f"{bytes_size:.2f} {unit}"
bytes_size /= 1024.0
def log_conversion(log_file_path, original_name, original_size, converted_name, converted_size):
"""Appends structural metadata into the timestamped session log."""
log_entry = (
f"Original File Name: {original_name}\n"
f"Original Size: {format_size(original_size)} ({original_size} bytes)\n"
f"Converted File Name: {converted_name}\n"
f"Converted Size: {format_size(converted_size)} ({converted_size} bytes)\n\n"
)
try:
with open(log_file_path, 'a', encoding='utf-8') as f:
f.write(log_entry)
except Exception as e:
print(f"⚠️ Failed to write to log file: {e}")
def scan_and_process(scan_path, max_files, log_file_path):
if not os.path.exists(scan_path):
print(f"❌ Error: The path '{scan_path}' does not exist or isn't mounted.")
return
if max_files == float('inf'):
print(f"\n🔍 Systematically scanning folder: {scan_path} (Processing ENTIRE folder)...")
else:
print(f"\n🔍 Systematically scanning path: {scan_path} (Batch Limit: {max_files} files)...")
print(f"📝 Logging session results directly to: {log_file_path}")
print("⏳ Scanning folders silently... Terminal will update when an H.264 file is found.")
converted_count = 0
for root, dirs, files in os.walk(scan_path):
# PROTECTION: Force the scanner to skip system trash dumps instantly if they exist in path
dirs[:] = [d for d in dirs if d.upper() not in ('$RECYCLE.BIN', 'SYSTEM VOLUME INFORMATION')]
for file in files:
if converted_count >= max_files:
print(f"\n🛑 Reached batch limit of {max_files} files converted. Stopping safely.")
return
if file.lower().endswith(VIDEO_EXTENSIONS):
file_path = os.path.join(root, file)
if "TEMP_TRANSCODING_" in file or "hevc" in file.lower() or "10-bit" in file.lower():
continue
try:
video_stream = get_video_stream_info(file_path)
if video_stream and isinstance(video_stream, dict):
codec = video_stream.get('codec_name', '')
width = video_stream.get('width', 0)
if codec == 'h264':
should_scale = width > 1920
if should_scale:
print(f"\n🎯 [Match] Codec: H.264 (4K). Downscaling to 1080p...")
else:
print(f"\n🎯 [Match] Codec: H.264. Keeping resolution...")
orig_name = os.path.basename(file_path)
orig_size = os.path.getsize(file_path)
temp_out, final_out = generate_arr_compliant_filename(file_path)
final_name = os.path.basename(final_out)
success = convert_to_h265_10bit(file_path, temp_out, scale_to_1080p=should_scale)
if success and os.path.exists(temp_out):
try:
conv_size = os.path.getsize(temp_out)
os.remove(file_path)
print(f"🗑️ Deleted original H.264 file: {file_path}")
os.rename(temp_out, final_out)
print(f"✅ Success! Managed File Finalized: {final_out}")
log_conversion(log_file_path, orig_name, orig_size, final_name, conv_size)
converted_count += 1
except Exception as e:
print(f"⚠️ Error while handling file swap/cleanup: {e}")
else:
if os.path.exists(temp_out):
os.remove(temp_out)
except Exception:
pass
if __name__ == "__main__":
print("🚀 Initializing NVENC Multi-Pass Transcoder with Logging Automation...")
if not os.path.exists(LOG_DIRECTORY):
try:
os.makedirs(LOG_DIRECTORY)
except Exception as e:
print(f"❌ Critical Error: Could not build path {LOG_DIRECTORY}. {e}")
sys.exit(1)
session_timestamp = datetime.now().strftime("%d-%m-%Y_%H%Mh")
current_log_file = os.path.join(LOG_DIRECTORY, f"{session_timestamp}.txt")
# 1. Prompt for drive selection
user_drive = input("💾 Enter the drive letter to scan (e.g., L, D, E): ").strip().upper()
# 2. Prompt for specific folder targeting selection
folder_choice = input("📁 Do you want to process a particular folder on this drive? (Y/N): ").strip().upper()
if folder_choice == 'Y':
target_path = input(f"📂 Paste the full folder path: ").strip()
max_files_to_convert = float('inf')
scan_and_process(target_path, max_files_to_convert, current_log_file)
print(f"\n🎉 Folder conversion run completed successfully!")
else:
target_path = f"{user_drive}:\\"
limit_input = input("🔢 How many H.264 files do you want to convert in this batch? ").strip()
if limit_input.isdigit() and int(limit_input) > 0:
max_files_to_convert = int(limit_input)
scan_and_process(target_path, max_files_to_convert, current_log_file)
print(f"\n🎉 Script batch run completed on Drive {user_drive}:\\")
else:
print("❌ Invalid number entered for batch limit.")
This has worked flawlessly for me. Save it in a file called "transcode_batch.py" in your C: drive in a folder called TaskSchedulerTasks (More on this later). In that folder also create a seperate folder named Transcoder Logs.
5) The fun part! In Powershell admin go to the TaskSchendulerTasks folder and run:
python transcode_batch.py
*WARNING* For a test, you may want to throw a single h.264 file on a drive to make sure you are happy with the conversion. Assuming you are, you can either start on the complete drive, or do a smaller batch (5, 10, 20, 50) or just choose a particlar season folder. No worries, the script will leave anything h.265 alone and only convert h.264 files. Keep in mind, this is tested on NVidia GPU's! I am using a 3090 for my conversions and a typical 43m TV show takes about 7m to transcode but turns a 4.6GB file into a 750MB file (about 84% size reduction). Doing a library of 100TB in a single run will take you literal weeks! So smaller portions are highly recommended and should only be done AFTER you've tried the sonarr/radarr filtering of content first. Also, you may want to move Anime or other Cartoon libraries somewhere where it won't see them. H.265 can makes stuff like that look awful. So maybe move them off a drive you are converting first.
6) If you have converted your entire library, there's only a single step left! AUTOMATION on future downloads!
This version of the script should be ran every once in a while. Only thing you'll need to change here is under configuration to change the drive letters to the drives/arrays that you are storing your media on. (I have 9x100TB Terramaster D5-310's arrays currently so that's what's there) Just simply change it to your requirements save this as automated_transcode_task.py in the same folder as the original file:
import os
import subprocess
import json
import re
from datetime import datetime
# ==================== CONFIGURATION ====================
# Targeted Hard Drive Array Roots
TARGET_DRIVES = [r"K:\\", r"L:\\", r"M:\\", r"N:\\", r"O:\\", r"P:\\", r"Q:\\", r"R:\\" r"S:\\"]
# Supported video extensions to check
VIDEO_EXTENSIONS = ('.mp4', '.mkv', '.avi', '.mov', '.m4v', '.flv', '.ts')
# Directory where the logs will be saved
LOG_DIRECTORY = r"C:\TaskSchedulerTasks\Transcode Logs"
# =======================================================
def get_video_stream_info(file_path):
"""Uses ffprobe to extract codec, resolution, and pixel format safely."""
cmd = [
'ffprobe',
'-v', 'error',
'-select_streams', 'v:0',
'-show_entries', 'stream=codec_name,pix_fmt,width,height',
'-of', 'json',
file_path
]
try:
result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=True)
data = json.loads(result.stdout)
if 'streams' in data and len(data['streams']) > 0:
return data['streams'][0] # Grab video stream index 0 safely
except Exception:
pass
return None
def generate_arr_compliant_filename(file_path):
"""Removes old resolution/codec tags and forces '1080p.HEVC.10-Bit'."""
directory, filename = os.path.split(file_path)
base, ext = os.path.splitext(filename)
# Regex to strip out old quality/codec metadata
pattern = re.compile(r'[-._\s]?(?:h[-._]?264|x264|avc|2160p|4k|1080p|720p)[-._\s]?', re.IGNORECASE)
cleaned_base = pattern.sub('.', base)
cleaned_base = re.sub(r'\.+', '.', cleaned_base).strip('.- ')
new_filename = f"{cleaned_base}.1080p.HEVC.10-Bit{ext}"
temp_output_path = os.path.join(directory, f"TEMP_TRANSCODING_{new_filename}")
final_output_path = os.path.join(directory, new_filename)
return temp_output_path, final_output_path
def convert_to_h265_10bit(input_path, output_path, scale_to_1080p=False):
"""
Executes FFmpeg utilizing NVENC hardware acceleration.
Enforces a strict bitrate ceiling to smash file sizes down to the ~600MB benchmark.
"""
cmd = ['ffmpeg']
if scale_to_1080p:
cmd.extend([
'-hwaccel', 'cuda',
'-c:v', 'h264_cuvid', # Force hardware decoding for H.264
'-resize', '1920x1080', # Resize directly during hardware decode step
'-i', input_path
])
else:
cmd.extend([
'-hwaccel', 'cuda',
'-i', input_path
])
cmd.extend([
'-c:v', 'hevc_nvenc', # Use NVIDIA RTX Hardware encoder circuit
'-pix_fmt', 'p010le', # Force 10-Bit color profile (smooths gradients/eliminates banding)
'-preset', 'p7', # Max quality NVENC encoding path math
'-cq', '28', # Deep efficiency quality target
'-maxrate', '2200k', # Strict video bitrate ceiling to cap data bloat
'-bufsize', '4400k', # Rate control tracking buffer
'-c:a', 'copy', # Pass audio tracks through untouched (Zero audio overhead)
'-c:s', 'copy', # Pass subtitle streams through untouched
output_path
])
try:
# Silenced outputs during background automation to protect logs
subprocess.run(cmd, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
return True
except subprocess.CalledProcessError:
return False
def format_size(bytes_size):
"""Formats file sizes into human-readable versions."""
for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
if bytes_size < 1024.0:
return f"{bytes_size:.2f} {unit}"
bytes_size /= 1024.0
def log_conversion(log_file_path, original_name, original_size, converted_name, converted_size):
"""Appends structural metadata into the timestamped session log."""
log_entry = (
f"Original File Name: {original_name}\n"
f"Original Size: {format_size(original_size)} ({original_size} bytes)\n"
f"Converted File Name: {converted_name}\n"
f"Converted Size: {format_size(converted_size)} ({converted_size} bytes)\n\n"
)
try:
with open(log_file_path, 'a', encoding='utf-8') as f:
f.write(log_entry)
except Exception:
pass
def main():
if not os.path.exists(LOG_DIRECTORY):
os.makedirs(LOG_DIRECTORY)
session_timestamp = datetime.now().strftime("%d-%m-%Y_%H%Mh")
log_file_path = os.path.join(LOG_DIRECTORY, f"AutoRun_{session_timestamp}.txt")
for root_dir in TARGET_DRIVES:
if not os.path.exists(root_dir):
continue
for root, dirs, files in os.walk(root_dir):
# PROTECTION: Force the scanner to skip system/recycle trash dumps instantly
dirs[:] = [d for d in dirs if d.upper() not in ('$RECYCLE.BIN', 'SYSTEM VOLUME INFORMATION')]
for file in files:
if file.lower().endswith(VIDEO_EXTENSIONS):
file_path = os.path.join(root, file)
if "TEMP_TRANSCODING_" in file or "hevc" in file.lower() or "10-bit" in file.lower():
continue
try:
video_stream = get_video_stream_info(file_path)
if video_stream and isinstance(video_stream, dict):
codec = video_stream.get('codec_name', '')
width = video_stream.get('width', 0)
if codec == 'h264':
should_scale = width > 1920
orig_name = os.path.basename(file_path)
orig_size = os.path.getsize(file_path)
temp_out, final_out = generate_arr_compliant_filename(file_path)
final_name = os.path.basename(final_out)
# Execute NVENC render pipeline
success = convert_to_h265_10bit(file_path, temp_out, scale_to_1080p=should_scale)
if success and os.path.exists(temp_out):
try:
conv_size = os.path.getsize(temp_out)
# Strict Safety: Only delete source on confirmed exit 0 code
os.remove(file_path)
os.rename(temp_out, final_out)
log_conversion(log_file_path, orig_name, orig_size, final_name, conv_size)
except Exception:
pass
else:
if os.path.exists(temp_out):
os.remove(temp_out)
except Exception:
pass
if __name__ == "__main__":
main()
7) Setting up scheduled tasks. Automate via Windows Task Scheduler
To make this script run unattended every couple of days in the background:
- Open Task Scheduler from your Windows Start Menu.
- Click Create Basic Task... in the right-side actions panel.
- Name it:
Automated Library Transcoderand hit Next. - Trigger: Select Weekly, set it to repeat every 1 week on your chosen days (e.g., Mon, Wed, Fri) at a time when your server is quiet (like 2:00 AM).
- Action: Select Start a program.
- Program/script: Type exactly:
python - Add arguments: Paste the exact directory path to your script:
C:\TaskSchedulerTasks\automated_transcode_task.py
Click Next, check the box for "Open the Properties dialog for this task when I click Finish" and hit Finish.
Crucial Security Settings (Properties Dialog):
- Under the General tab, select Run whether user is logged on or not.
- Check the box at the bottom for Run with highest privileges (This gives Python the admin permissions required to delete original files and swap filenames across array roots cleanly).
And THAT'S IT! From now on all new media regardless of what you download will automatically convert to H.265 1080p 10-bit. It will automatically compress everything new on your schedule!
----- GAMES ----
Next is going to be GAMES. I have a vast Steam Library (Over 1000 games) and what I absolutely hate doing is having to uninstall anything to make room for something else. Hard drives don't really matter, but NVMe storage is at a premium! Since so many games work much better on NVMe's, I am using a program called CompactGUI. What it does is scan your installed games and compresses whatever assets are not well optimized. You mileage may vary on how well it compresses it. I found that highly optimized titles are already highly optimized and may only compress 5%. Many indy games or smaller titles are very poorly optimized and I was able to reduce 10-15GB down to 3-5GGB. Over an entire NVMe drive with mixed game content, I was averaging 30% reduction in file sizes which is astoundingly good! A small little word of warning... If you're running a good processor (i7-10700k/Ryzen 3800x) or higher, LZX is what you want for maximum compression. For lower-end CPU's will need to run the smaller compression models. On the CPU's I mentioned and better there's a almost non-existant performance deficit. Maybe 1-2 frames and a few % points of CPU spikes.
Step 1: Open Terminal as Administrator
Press the Windows Key + X and select Terminal (Admin), PowerShell (Admin), or Command Prompt (Admin). Running as administrator ensures Windows won't block you from modifying protected game or program folders. [1, 2]
Step 2: Use the Compact Command
The standard syntax to compress a folder using Windows' modern, high-efficiency algorithms is:
cmd
compact /c /s:"C:\Path\To\Your\Folder" /a /i /exe:ALGORITHM
Replace ALGORITHM with one of the four types used by CompactGUI:
XPRESS4K(Fastest, lower compression)XPRESS8K(Balanced - This is CompactGUI's default)XPRESS16K(Slower, higher compression ratio)LZX(Slowest, highest compression – Recommended for games on SSDs)
Example: Compressing a Steam Game
If you wanted to compress Arma 3 using the high-ratio LZX algorithm, you would run:
cmd
compact /c /s:"C:\Program Files (x86)\Steam\steamapps\common\Arma 3" /a /i /exe:lzx
Explanation of the Terminal Flags:
- /c – Instructs Windows to compress the specified files.
- /s – Tells it to apply the command to the selected folder and all of its subfolders.
- /a – Includes hidden and system files in the compression process.
- /i – Ignores errors (such as files that are actively locked or in use) and keeps compressing the rest.
- /exe – Force-enables the multi-threaded Windows 10/11 algorithms, which compress files transparently so they remain fully executabl
Now the fun part! AUTOMATION! Depending on the speed of your drives, and the amount of drives you have, this could be a quick, or slow, process. A fast drive operating at peak efficiency can chew through 1TB in less than an hour. Older NVMe's or systems where you aren't well optimized, it could take much longer. But once you've done all your drives, the only thing it will do is new files! You can easily set this up in Task Scheduler with a simple batch file:
@echo off
:: ==========================================
:: C DRIVE TARGETED FOLDERS
:: ==========================================
compact /c /s:"C:\Program Files (x86)\Steam\steamapps\common" /i /exe:lzx
compact /c /s:"C:\Games" /i /exe:lzx
:: ==========================================
:: ENTIRE DEDICATED GAME DRIVES
:: ==========================================
compact /c /s:I:\ /i /exe:lzx
compact /c /s:W:\ /i /exe:lzx
compact /c /s:X:\ /i /exe:lzx
compact /c /s:Y:\ /i /exe:lzx
compact /c /s:Z:\ /i /exe:lzx
exit
This is my example. I have 5x 2TB NVMe's (I, W, X, Y, Z) and then my main C: I just want it to look at Steam and my Games folders (Games folder is Non-Steam). Once you've completed the initial compression, this batch file will automatically scan the drives/folders you put in here and any new content is auto-compressed to free up space. I have this running under task scheduler in windows to check once every week during times no one is using the server.
Here is how to set up Windows Task Scheduler to run your batch script automatically with the elevated admin rights it needs to compress entire drives.
Step 1: Open Task Scheduler as Administrator
- Press the Windows Key, type Task Scheduler, right-click it, and select Run as administrator.
Step 2: Create a New Basic Task
- Look at the Actions panel on the far-right side of the window.
- Click Create Basic Task...
- Name it something clear, like
Game Drive Auto-Compression. - Add a description if you'd like (e.g., Runs compact.exe on game directories), then click Next.
Step 3: Set Your Schedule (Trigger)
- Choose how often you want this to run (e.g., Weekly or Monthly is usually best for compression routines, as running it daily on entire drives is overkill). Click Next.
- Set the specific Start Time and day of the week.
- Tip: Choose a time when your PC is turned on but you aren't gaming, like a Sunday morning or late at night. Click Next.
Step 4: Link Your Script (Action)
- Select Start a program and click Next.
- Click the Browse... button next to the "Program/script" box.
- Navigate to and select your
.batfile. - Crucial Rule for Paths: In the Start in (optional) box, paste the path to the folder where your batch file lives without quotes (e.g.,
C:\Scripts). This ensures Windows doesn't get confused about where to look for files when running in the background. Click Next, then click Finish.
Step 5: Grant Highest Privileges (Crucial!)
Because your script modifies root directories (I:\, W:\, etc.), Task Scheduler will silently fail unless you grant it admin authority.
- In the middle pane of Task Scheduler, click on Task Scheduler Library.
- Find your new task in the list, right-click it, and choose Properties.
- Under the General tab, look at the bottom and check the box that says "Run with highest privileges".
- (Optional) Change the dropdown for "Configure for:" to Windows 10 or Windows 11 depending on your OS.
Step 6: Prevent Laptop Battery Drain (If Applicable)
- Switch to the Conditions tab at the top.
- Under the Power section, ensure "Start the task only if the computer is on AC power" is checked so it doesn't max out your CPU and drain your battery if you are away from an outlet.
Click OK to save everything.
THAT'S IT! You officially free'd up a massive amount of storage while maintaining your entire library allowing you to get even more backups! The ultimate goal of a datahoarder on a storage constraint! Hopefully this may help some of you. For those of you just starting out, this is a wonderful automation system that can be used right away to keep you drives as densly packed with content as possible almost immediately!
r/DataHoarder • u/PrivacyStack • 17h ago
Question/Advice Car Talk Archive?
I hope this is okay to post. Car Talk was a show entirely broadcast on public radio. As a Father’s Day gift, I would like to give my Dad access to as much of the history of the show as possible. I found a torrent, but no one is seeding it. Does anyone know where I can find the entire show’s archive or where I might be able to archive it myself?
r/DataHoarder • u/StarLegacy1214 • 1d ago
Discussion Archive.today is down (Too Many Requests)
Yeah, it’s been down since yesterday. Anytime I go on their website, all I get is “Too Many Requests”. Switching to .ph and .is and even .md leads to the same thing. I can’t contact the site owner because I can’t find their e-mail, plus it seems their blog hasn’t been updated since March. Seems no one else is saying anything either?
r/DataHoarder • u/TidderUsername123 • 1d ago
Question/Advice Has anyone heard their hard drive make this sound?
Enable HLS to view with audio, or disable this notification
r/DataHoarder • u/MK2809 • 1d ago
Backup Anywhere to buy cheaper external HDDs from? UK based.
I'm a videographer and the cost of new storage is really hurting. I like to have 2 backups of all shoots, but with how storage has more than doubled since mid 2025, it's becoming more of a challenge.
Is there anywhere to buy cheaper storage from? Is there anywhere reliable for used or refurbished that is less expensive than brand new?
Thanks
r/DataHoarder • u/Boring_Occasion_3526 • 1d ago
Discussion Phone photos are the weak link in my 3-2-1 — how do you handle the phone leg?
My NAS / external drive setup is solid for my computer files, but phone photos are the messy part and I keep going back and forth on how to count them.
Right now I have:
- Phone + cloud sync (which feels like 1.5 copies, not 2, since a delete / sync issue can propagate)
- A scheduled local copy to an external drive every week or two
- The local copy is just a big dump by date, so I lose the album organization I had on the phone
What I'm stuck on:
1. Do you count cloud sync as a real copy in your 3-2-1, or only as a convenience copy?
2. How do you handle the phone leg — scheduled local copy, manual export, something else?
3. When you export, do you preserve albums / folders, or just accept a flat date dump?
4. Do you actually verify the local copy — spot-check opens, checksums, test restore?
Curious how other people do this, especially for family phones where you have 2-3 phones feeding into one backup.
r/DataHoarder • u/bookpreservation26 • 1d ago
Question/Advice Best UK company to digitise a rare book
I have a book on the way, one which was cancelled last minute by the publishers due to uncovered behavior from the author in their last, some copies managed to escape to the wild and I've got one with the idea of getting it scanned, I don't endorse the author but it has art and forwards to sections from many talented people that collaborated on this project and my thoughts are it is out there why not archive it?
I am aiming for the highest quality scans possible which after some research means cutting the spine and scanning and I am fine with this but there are a few options in the UK and I am wondering if anyone has any recommendations for who to use for this?
r/DataHoarder • u/DocWatson42 • 2d ago
News News item: "The Wayback Machine Preserved 21 Years of Photos from a Webcam That Deletes Everything"
Hein, Rich (1 October 2026). "The Wayback Machine Preserved 21 Years of Photos from a Webcam That Deletes Everything". How-To Geek.
This piece includes a technique that is of potential interest to data hoarders. (I apologize if it is already known.)
r/DataHoarder • u/mrcelophane • 1d ago
Question/Advice Looking to host some Pro Wresling Videos and looking for suggestions on app. StashApp?
Unconventionally, a SFW Stash App instance seems like a great choice? The app already has a way to filter by performer/wrestler, can use Markers to notate start and end time of different matches and segments, etc.
But what else is out there? Any other data marks out there that are already solving this problem?
r/DataHoarder • u/garden-plant • 1d ago
Backup Request for someone to save a backup of the Flora of North Dakota Database
The Flora of North Dakota Database was created by a now-deceased Russian botanist who taught in North Dakota for nine years. It is entirely in the public domain.
The Internet Archive has a copy of it, but I'm not sure how easy it would be to reconstitute the original database from the copy on the archive. There are thousands of photos. The maps no longer work due to the server not having the software necessary to read the code for generating the maps.
I cannot post the url due to the domain extension being not allowed, you can find it with a search.
The purpose of a backup would be in case something happens to the only online copy.
This is an open request for anyone interested in copying the database and backing it up, without hosting being necessary.