@rvstaveren there’s a way to do it by putting a file on the boot media, check out the recovery.bin mentions in here: https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#raspberry-pi-boot-eeprom
Or image a fresh SD card with Linux and use that for a one-time boot.