(BG95) how to properly parse QHTTPREAD response when partial data?

Hello.

Situation: (lengthy) HTTP file download. the QHTTPGET URC tells me how many bytes to expect after the CONNECT response of the QHTTPREAD.

What if the download is interrupted for some reason, e.g. no reception, how am I supposed to know that the in-band data bytes (file content) have stopped prematurely? I would have to look for the URC in what is supposed to be file content… which may possibly contain the same string as data (not a URC).

How does Quectel expect customers to handle this?

when you do Qhttpget operation , if you have not received QHTTPREAD response ,and meet one err ( timer time out) , the receiving buffer will be cleans automatcally .

costomers have to re-do Httpget cmd again

And make sure that the pDp context has been activated successfully

I’ll rephrase.

the download happens. I receive data. however, if something interrupts the download, such as the modem losing reception, or the remote server closing the connection or crashing, I will receive some data, but not all data.

>  "AT+QHTTPURL=36" <CR>
<  "AT+QHTTPURL=36" <CR>
<  <CRLF> "CONNECT" <CRLF>
>  "http://www.mydomain.com/somefile.bin"
<  <CRLF> "OK" <CRLF>
>  "AT+QHTTPGET" <CR>
<  "AT+QHTTPGET" <CR>
<  <CRLF> "OK" <CRLF>
... half a second ...
<  <CRLF> "+QHTTPGET: 0,200,131072" <CRLF>
>  "AT+QHTTPREAD" <CR>
<  "AT+QHTTPREAD" <CR>
<  <CRLF> "CONNECT" <CRLF>
<  [data, but less than 131072 bytes...]
<  <CRLF> "OK" <CRLF>
<  <CRLF> "+QHTTPREAD: 719" <CRLF>

the problem is that there may be fewer bytes than promised (by +QHTTPGET URC) in case the connection is interrupted.

do you understand the problem? if I simply read as many bytes as announced, but less than that arrive, I read the OK and +QHTTPREAD URC as data, when they aren’t data but signaling.

to detect this situation, I would have to look for the URC string in the data… but the data might contain that string itself!

that is unacceptable.

the HTTP(S) application note did not address this situation. it needs to be addressed however.

the modem’s AT protocol responses should be extended such that no assumption needs to be made about how many bytes of payload actually follow after a CONNECT.

my proposal: the modem sends pieces of whatever size, that have actually been received by the modem and reside in the modem’s buffer, and sends them across the UART using this syntax or a similar one:

  • <CRLF> "CONNECT 1234" <CRLF> [1234 bytes...]
  • repeat as long as there is data
  • when no more data: <CRLF> "OK" <CRLF>

does this sound reasonable?

Hi, did you solve this problem @crackwitz?

No, I could not. We abandoned our investigation of QHTTPREAD in BG95.

If you encounter this issue yourself, you should take it up with Quectel.

The QHTTPREAD response is broken, as I described previously. The stream is sent without prefixed length, without chunking, without escaping of termination symbols (error messages). This is simply bad protocol design. I think the QHTTPREAD feature was new back then and had not received any user feedback or review by senior software engineers. It would have to be reengineered. If I’d had access to the firmware sources of the BG95, I might have done it myself.