Non-standard BaudRates don't work when uploading with AVRdude

[Virtual-Java]

After I failed at uploading code to an avr optiboot like bootloader written in assembly by bitflipser, I tried to upload to a atmega328p containing Optiboot compiled with a NON-standard baud rate.
Both were built for 250000 baud and in both cases uploading with AVRdude failed.
Btw. only the 115200 Baud version of ASMboot worked for me, asmOptiboot not. (https://github.com/bitflipser/ASMboot vs. https://github.com/bitflipser/asmOptiboot)

So I decided to test Optiboots support of different baud rates: 38400, (111111), 115200, (125k), 250k, 500k, 1M baud.
For all of my tests I used the following three settups:
1. Orig. Arduino Uno@16 MHz, 5V, (Original DFU bootloader on atmega16u2)
2. Hoodloader Uno@16 MHz, 5V, (Hoodloader2.0.5 bootloader on atmega16u2)
3. AVR Standalone@16 MHz, 5V, (RS232toUART, TTL-level converter MAX232), Upload via commercial USBtoRS232 and manual reset (no DTR)
To get plausible test results I used a AtMega328p in a PDIP package which I moved the between the sockets and the breadboard.

For a complete test log including the whole output of avrdude see: https://github.com/Virtual-Java/optiboot-usb-mcus/blob/avrdude-test/optiboot/hexfiles/optiboot_custom/Optiboot_avrdude_upload.txt
A short summary of all the commands I used can be found here: https://github.com/Virtual-Java/optiboot-usb-mcus/blob/avrdude-test/optiboot/hexfiles/optiboot_custom/Optiboot_avrdude_upload_short.txt

Taking a look at the "makeall.mcudude" script created by MCUdude baud rates greater than 115200 e.g. 250k, 500k, (1M) should be possible under certain circumstances. Indeed this script compiles for different target mcus, but they support the same clock speeds. Unfortunately I wasn't able to upload at non-standard baud rates.
Nevertheless the only baud rates that worked for me were 38400 and 115200 baud.

Neither speeds a bit lower (111111 baud) nor a bit higher (125000 baud) enabled uploading successfully. However the difference to the desired baud rate is 0.0% in both cases compared to 2.1% at 115200 baud.
Since the file is distributed with the official optiboot, I assume that other people were able to upload at higher speed.
https://github.com/MCUdude/optiboot_flash

Can you tell me if upload using non-standard baud rates fails due to a bug in AVRdude (Version 6.3-20190619), did I do something wrong or is a solar wind currently interfering my communication? :)

[WestfW]

You COULD be running into problems with components OTHER than the bootloader.
I'm not sure if all USB/Serial converters support arbitrary bit rates. Some operating systems might not (also depending on which sort of library access they are using for controlling the serial port.)
A MAX232 chip is typically only rated to work up to 120kbps, and at that sort of speed your wiring also becomes much more critical.

[Virtual-Java]

Thanks a lot, your tip was great. The operating system I used (Linux Mint) didn't support non-standard baud rates by default.
After compiling a small c program to setup custom baud rates, I was able to upload.
https://github.com/GrantEdwards/Linux-arbitrary-baud
User@PC:~/path/to/github/repository/Linux-arbitrary-baud$ ./arbaud-Wall /dev/ttyACM0 333333 ==> actual speed reported 333333

Maybe this is good to know for other users too, since it results in issues which are hard to debug. arduino/Arduino#3351

To see if there are benefits of greater upload speeds I evaluated the duration required for programming the flash memory at different upload speeds. https://github.com/Virtual-Java/optiboot-usb-mcus/blob/avrdude-test/optiboot/hexfiles/optiboot_custom/UploadSpeedCompare.txt

Summary:
  115200 baud Ø: r: 0.00s; w: 6.49s; v: 4.39s
  250000 baud Ø: r: 0.01s; w: 5.45s; v: 3.35s
  500000 baud Ø: r: 0.01s; w: 4.43s; v: 2.35s
1000000 baud Ø: r: 0.01s; w: 4.44s; v: 2.40s
2000000 baud Ø: r: 0.00s; w: 4.40s; v: 2.37s

Uploading at baud rates greater than the standard 115.2 kbaud is a bit faster, but speed does not behave proportional to frequency. e.g. uploading at with 250 kbaud, which is more than two times the standard baud rate, saves 2 seconds only. This equals approximately -18%.
Up to baud rates of 500000 the total duration decreases a bit, but increasing it further has no advantages and could even have the opposite effect. At high baud rates the time required to program the flash cells outweighs that spent for communication.
Furthermore non-standard in especially high baud rates, may result in errors which need more time to debug than you ever save.

On some platforms the user is forced to adjust settings manually, install special tools to setup custom baud rates
and can come into conflicts with unsupported hardware like serial ports or converters supporting a maximum speed of 115200 Baud.
With MAX232 a reliable connection was possible until 250 kbaud, though this is more than twice the value specified.
In my tests 666666 baud was the absolute maximum possible with MAX3232, but it's strongly not recommended to use such high baud rates for commercial designs!!!
https://github.com/Virtual-Java/optiboot-usb-mcus/blob/avrdude-test/optiboot/hexfiles/optiboot_custom/Optiboot_avrdude_upload2_custom_baudrates_work.txt

Consequently non-standard baud rates should be used for testing purposes only. For production it is strongly recommended
to use one of the more reliable standard baud rates (300, 1200, 2400, 4800, 9600, 19200, 38400, 57600, 115200).

One of the benefits of programming at baud rates of 250 kbaud (linux-non-standard) and 500 kbaud (linux-standard) is the very small difference between the desired and the real baud rate. Subsequently the question arises whether a baud rate being as exactly as possible or a lower speed has more influence on sufficient reliability.

However uploading at these high baud rates is really fast,
but will there still be enough time left to to drink a coffee?

[WestfW]

Thankyou for the detailed update! It's hard to keep track of which OSes and OS versions support arbitrary bitrates on their "serial" ports.

According to the datasheet, a flash page write takes between 3.7 and 4.5ms, and the page size on an ATmega328p is 128 bytes or 1024 bits. that works out to a speed of about 250kbps just for the programming, which is about what you observed.

Optiboot USED to have code that would overlap the programming of one page with the reception of the next page, but it was more complicated and bigger, and I decided it wasn't worth it and removed it to save space. Back in 2014 for version 6. Programming a full 328p got about 1s slower.