[BG96] QHTTPPOST Error 719

IoT Modules LPWA Module
1

I’ve been using the BG96MA for a while now and every now and then I have problems with the commands AT+QHTTPPOST and AT+QHTTPPOSTFILE. With some combinations of firmware version, POST command, server and payload I always get an error 719 (HTTP socket closed). For example: I’ve been using firmware version BG96MAR02A07M1G_01.016.01.016 and the AT+QHTTPPOST command without a problem for months. But now I have modules with version BG96MAR03A06M1G_01.005.01.005 and the command practically always returns error 719. After using the AT+QHTTPPOSTFILE command, most of the connections work with the newer firmware, but modules with an older firmware now return error 719.

I managed to find a minimal example that shows the error with firmware versions BG96MAR02A07M1G_01.011.01.011 and BG96MAR03A06M1G_01.005.01.005. The error can be reproduced on custom hardware/firmware and on a BG96 Eval-Kit using QNavigator. If I use following commands one request succeeds, while the other fails. I omitted commands that deal with the connection to the GPRS network, since the connection obviously works. The only difference between both requests, is that one contains an authorization token and the other does not.

# Setup connection
>>> AT+CFUN=1
>>> AT+CGDCONT=1,"IPV4V6","TM"
>>> AT+COPS=0,0,"",0
>>> AT+QHTTPCFG="requestheader",1
>>> AT+QICSGP=1,1,"TM"
>>> AT+QIACT=1

# This request succeeds
>>> AT+QHTTPURL=24,120
>>> https://httpbin.org/post
>>> AT+QFDEL="HTTPPOST.TMP"
>>> AT+QFUPL="HTTPPOST.TMP",1554,120
### ... [Upload file HTTP_ok.TMP from bg96_error_719.zip]
>>> AT+QHTTPPOSTFILE="HTTPPOST.TMP",120

# This request fails
>>> AT+QHTTPURL=24,120
>>> https://httpbin.org/post
>>> AT+QFDEL="HTTPPOST.TMP"
>>> AT+QFUPL="HTTPPOST.TMP",2061,120
### ... [Upload file HTTP_fail.TMP from bg96_error_719.zip]
>>> AT+QHTTPPOSTFILE="HTTPPOST.TMP",120

I tried POST requests with multiple HTTP servers (Apache, Flask, Gunicorn) and this error can be seen on any of them. When running a simple Flask server, it actually seems like all data is being received correctly and the server sends a correct HTTP response. But somehow this gets mixed up with the data being send. A Wireshark trace on the server shows, that the request and the response get mixed up and some of the data is rejected with a Reset-Flag (RST).

server_trace
server_trace1375×503 72 KB

What exactly leads to this error and how can I fix my HTTP POST requests? The data to reproduce the error can be found in following zip file.

bg96_error_719_data.zip (3.2 KB)

2

I think I narrowed down the problem. If I try to send a file via HTTPS post request, it fails with error 719, if the 12th bit of the total length (including all headers) is set. So for example: sending a request with 2047 bytes total works, while a request with 2048 bytes fails. And just like that 4095 bytes fails, while 4096 bytes succeeds. This is 100% reproducible on custom hardware and the development kit. My original test files from the first post also show this behavior (1554 bytes OK, 2061 bytes fail). To clarify some more examples:

2047  ==> OK
2048  ==> Fail
4095  ==> Fail
4096  ==> OK
6143  ==> OK
6144  ==> Fail
8191  ==> Fail
8192  ==> OK
10239 ==> OK
10240 ==> Fail
12287 ==> Fail
12288 ==> OK
14335 ==> OK
14336 ==> Fail

As a workaround I’m now padding the HTTP header to some valid length. But it would be great if there would be a real explanation and solution for this. Since this only happens with HTTPS connections, I would guess that there is a problem with a buffer in the SSL stack (buffer size probably 2048 or 4096 byte).

3

Hello, I am experiencing this problem also.

How did you implement your “workaround”; how did you determine the byte count that would be sent and what did you add to it?

4

I’m basically calculating the size of the complete request and adding the header field “X-PADDING: ZZZZ…” to fill it up to a “good” size. Unused header fields are supposed to be ignored by clients and servers, so the field only increases the size of the request without disturbing the communication. Counting the bytes is a bit fiddly, but I end up with something like this:

POST /post HTTP/1.1
Host: httpbin.org
User-Agent: Mozilla/5.0
Content-Type: application/octet-stream
X-PADDING: ZZZZZZZZ
Content-Length: 3954

[DATA TO BE SEND]

I don’t have real data on hand so it’s just an example and the numbers might not fit perfectly, but basically without the X-PADDING header the complete request would be something like 4076 bytes long (3954 content + header) and would fail. The padding adds exactly 20 bytes, so the total request is now 4096 bytes long and will send without a problem.

I only add the padding if a problematic firmware version is detected (i.e. R03A06) and the calculation of the padding length works as follows:

static size_t workaround_calc_padding_len(size_t expected_len)
{
uint32_t ceil_value;
if (expected_len & 2048)
{
ceil_value = 2048 * ((expected_len / 2048) + 1);
return ceil_value - expected_len;
}
else
{
return 0;
}
}

The rest I can’t really share, but you basically have to estimate the length of the request. So after writing the Content-Type header I calculate (header_size + "Content-Length: " + log10(data_size) + data_size) and add the correct size padding.

Of course, if you use a tolerant data format you also could pad the actual data. With json, for example, you could just add spaces to the end without changing the actual data.

5

Hello, I am also experiencing this problem. It seems like the last packet of bigger data chunk lacks behind others. It always arrives after server responds, as shown in wireshark screenshot above. Is there any new workaround other than header padding ?. Im using BG96MAR04A02M1G_02.002.01.002.