[MC60] GSM SSL TCP cannot detect if server is closed

[webyfymad]

Sir/Mam,

We have been testing data transmission using SSL. We noticed that AT+QSSLSEND mostly returns SEND OK even if the server process was stopped in the remote.

We do not seem to receive any URC notifying of disconnection from the server.
AT+QSSLSTATE also always return CONNECTED.

This is after removing the antenna from the evaluation kit to simulate network loss:

+QSSLSTATE: PDP DEACT

+QSSLSTATE: 0,“”,“”,“INITIAL”,0

+QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1

+QSSLSTATE: 2,“”,“”,“INITIAL”,0

+QSSLSTATE: 3,“”,“”,“INITIAL”,0

+QSSLSTATE: 4,“”,“”,“INITIAL”,0

+QSSLSTATE: 5,“”,“”,“INITIAL”,0

OK

Kindly look into this.

Regards

[Kyson]

Dear Webyfymad,
Thanks for your inquiry in Quectel forum.
Normally, if the tcp server have close the connection, the module will have URC reported when the module receive such information.

Just as you have said that you cannot receive the disconnection URC, it may need you provide more information to analyze the reason. Please help to provide the whole test AT log, and marked the time when the server close the connection. Then we will help you analyze where is the problem. Thanks!

[webyfymad]

Hi Kyson,
Thank you for your reply. I am attaching two sets of logs.

05-03-2020 14:31:07 Sent AT
05-03-2020 14:31:07 Received AT
05-03-2020 14:31:07 Received OK
05-03-2020 14:31:07 Received
05-03-2020 14:31:09 Sent ATE0
05-03-2020 14:31:09 Received ATE0
05-03-2020 14:31:09 Received OK
05-03-2020 14:31:09 Received
05-03-2020 14:31:17 Sent AT+QIFGCNT=0
05-03-2020 14:31:17 Received
05-03-2020 14:31:17 Received OK
05-03-2020 14:31:17 Received
05-03-2020 14:31:23 Sent AT+QICSGP=1
05-03-2020 14:31:24 Received
05-03-2020 14:31:24 Received OK
05-03-2020 14:31:24 Received
05-03-2020 14:31:24 Received
05-03-2020 14:31:24 Received Call Ready
05-03-2020 14:31:24 Received
05-03-2020 14:31:28 Received
05-03-2020 14:31:28 Received SMS Ready
05-03-2020 14:31:28 Received
05-03-2020 14:31:30 Sent AT+QIREGAPP
05-03-2020 14:31:30 Received
05-03-2020 14:31:30 Received OK
05-03-2020 14:31:30 Received
05-03-2020 14:31:36 Sent AT+QIACT
05-03-2020 14:32:07 Received
05-03-2020 14:32:07 Received O
05-03-2020 14:32:07 Received K
05-03-2020 14:32:07 Received
05-03-2020 14:32:24 Sent AT+QSSLCFG=“sslversion”,0,4;+QSSLCFG=“seclevel”,0,2;+QSSLCFG=“ignorertctime”,1;+QSSLCFG=“cacert”,0,“NVRAM:CA0”;+QSSLCFG=“clientcert”,0,“NVRAM:CC0”;+QSSLCFG=“clientkey”,0, “NVRAM:CK0”
05-03-2020 14:32:24 Received
05-03-2020 14:32:24 Received OK
05-03-2020 14:32:24 Received
05-03-2020 14:32:38 Sent AT+QSSLOPEN=1,0,“xx.xx.xx.xx”,8080,0
05-03-2020 14:32:38 Received
05-03-2020 14:32:38 Received OK
05-03-2020 14:32:38 Received
05-03-2020 14:32:43 Received
05-03-2020 14:32:43 Received +QSSLOPEN: 1,0
05-03-2020 14:32:43 Received
05-03-2020 14:33:02 Sent AT+QSSLSTATE
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: IP PROCE
05-03-2020 14:33:02 Received SSING
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:33:02 Received
05-03-2020 14:33:02 Received OK
05-03-2020 14:33:02 Received
05-03-2020 14:33:19 Sent AT+QSSLSEND=1,12
05-03-2020 14:33:19 Received
05-03-2020 14:33:19 Received >
05-03-2020 14:33:25 Received
05-03-2020 14:33:25 Received SEND OK
05-03-2020 14:33:25 Received
05-03-2020 14:33:29 Sent AT+QSSLSEND=1,12
05-03-2020 14:33:29 Received
05-03-2020 14:33:29 Received >
05-03-2020 14:33:43 Received
05-03-2020 14:33:43 Received SEND OK
05-03-2020 14:33:43 Received
05-03-2020 14:33:53 Received
05-03-2020 14:33:53 Received +QSSLURC: “closed”,1
05-03-2020 14:33:53 Received
05-03-2020 14:35:08 Sent AT+QSSLOPEN=1,0,“xx.xx.xx.xx”,8080,0
05-03-2020 14:35:08 Received
05-03-2020 14:35:08 Received OK
05-03-2020 14:35:08 Received
05-03-2020 14:35:13 Received
05-03-2020 14:35:13 Received +QSSLOPEN: 1,0
05-03-2020 14:35:13 Received
05-03-2020 14:35:15 Sent AT+QSSLSEND=1,12
05-03-2020 14:35:15 Received
05-03-2020 14:35:15 Received >
05-03-2020 14:35:21 Received
05-03-2020 14:35:21 Received
05-03-2020 14:35:21 Received SEND OK
05-03-2020 14:35:21 Received
05-03-2020 14:35:35 Sent AT+QSSLSEND=1,12
05-03-2020 14:35:35 Received
05-03-2020 14:35:35 Received >
05-03-2020 14:35:37 Received
05-03-2020 14:35:37 Received SEND OK
05-03-2020 14:35:37 Received
05-03-2020 14:35:44 Sent AT+QSSLSEND=1,12
05-03-2020 14:35:44 Received
05-03-2020 14:35:44 Received >
05-03-2020 14:35:47 Received
05-03-2020 14:35:47 Received SEND OK
05-03-2020 14:35:47 Received
05-03-2020 14:35:48 Sent AT+QSSLSEND=1,12
05-03-2020 14:35:49 Received
05-03-2020 14:35:49 Received >
05-03-2020 14:35:51 Received
05-03-2020 14:35:51 Received SEND OK
05-03-2020 14:35:51 Received
05-03-2020 14:35:54 Sent AT+QSSLSEND=1,12
05-03-2020 14:35:54 Received
05-03-2020 14:35:54 Received >
05-03-2020 14:35:56 Received
05-03-2020 14:35:56 Received SEND OK
05-03-2020 14:35:56 Received
05-03-2020 14:36:14 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:14 Received
05-03-2020 14:36:14 Received >
05-03-2020 14:36:16 Received
05-03-2020 14:36:16 Received SEND OK
05-03-2020 14:36:16 Received
05-03-2020 14:36:21 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:21 Received
05-03-2020 14:36:21 Received >
05-03-2020 14:36:22 Received
05-03-2020 14:36:22 Received SEND OK
05-03-2020 14:36:22 Received
05-03-2020 14:36:24 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:24 Received
05-03-2020 14:36:24 Received >
05-03-2020 14:36:25 Received
05-03-2020 14:36:25 Received SEND OK
05-03-2020 14:36:25 Received
05-03-2020 14:36:27 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:27 Received
05-03-2020 14:36:27 Received >
05-03-2020 14:36:28 Received
05-03-2020 14:36:28 Received SEND OK
05-03-2020 14:36:28 Received
05-03-2020 14:36:30 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:30 Received
05-03-2020 14:36:30 Received >
05-03-2020 14:36:31 Received
05-03-2020 14:36:31 Received SEND OK
05-03-2020 14:36:31 Received
05-03-2020 14:36:32 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:32 Received
05-03-2020 14:36:32 Received >
05-03-2020 14:36:33 Received
05-03-2020 14:36:33 Received SEND OK
05-03-2020 14:36:33 Received
05-03-2020 14:36:34 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:34 Received
05-03-2020 14:36:34 Received >
05-03-2020 14:36:35 Received
05-03-2020 14:36:35 Received SEND OK
05-03-2020 14:36:35 Received
05-03-2020 14:36:35 Received
05-03-2020 14:36:37 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:37 Received
05-03-2020 14:36:37 Received >
05-03-2020 14:36:38 Received
05-03-2020 14:36:38 Received SEND OK
05-03-2020 14:36:38 Received
05-03-2020 14:36:39 Sent AT+QSSLSEND=1,12
05-03-2020 14:36:39 Received
05-03-2020 14:36:39 Received >
05-03-2020 14:36:42 Received
05-03-2020 14:36:42 Received SEND OK
05-03-2020 14:36:42 Received

As you see the URC did come the first time I stopped the server, after which I restarted the server and reconnected the MC60 to the server (at 14:35:08).

Then I sent some data (till 14:36:42) and kept the device Idle for some time, no other URC was received at this point. After which I stopped the server again, this time no URC came.

This is after stopping the server:

05-03-2020 14:41:53 Sent AT+QSSLSTATE
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: IP PROCESSING
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:41:53 Received
05-03-2020 14:41:53 Received OK
05-03-2020 14:41:53 Received
05-03-2020 14:41:58 Sent AT+QSSLSTATE
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: I
05-03-2020 14:41:58 Received P PROCESSING
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:41:58 Received
05-03-2020 14:41:58 Received OK
05-03-2020 14:41:58 Received
05-03-2020 14:42:05 Sent AT+QSSLSTATE
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: IP PROCESSING
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:42:05 Received
05-03-2020 14:42:05 Received OK
05-03-2020 14:42:05 Received
05-03-2020 14:42:07 Sent AT+QSSLSTATE
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: IP PROCESSING
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:42:07 Received
05-03-2020 14:42:07 Received OK
05-03-2020 14:42:07 Received
05-03-2020 14:42:19 Received
05-03-2020 14:42:19 Received >
05-03-2020 14:42:27 Received
05-03-2020 14:42:27 Received SEND OK
05-03-2020 14:42:27 Received
05-03-2020 14:42:30 Received
05-03-2020 14:42:30 Received >
05-03-2020 14:42:42 Received
05-03-2020 14:42:42 Received SEND OK
05-03-2020 14:42:42 Received
05-03-2020 14:44:47 Sent AT+QSSLSTATE
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: IP PROCESSING
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 0,“”,“”,“INITIAL”,0
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 1, “TCP”, “xx.xx.xx.xx”, 8080,“CONNECTED”,1
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 2,“”,“”,“INITIAL”,0
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 3,“”,“”,“INITIAL”,0
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 4,“”,“”,“INITIAL”,0
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received +QSSLSTATE: 5,“”,“”,“INITIAL”,0
05-03-2020 14:44:47 Received
05-03-2020 14:44:47 Received OK
05-03-2020 14:44:47 Received

This is a serious problem as we loose some messages. Kindly check this.

NB: In the second log before every (>) there is a AT+QSSLSEND=1,12 called. It was accidentally deleted from the log

[Kyson]

Dear Webyfymad,
Thanks for your updating.
Just as you have said that after you close the server at about 14:36:42, but the module have no URC information to reminding you, while it still said that the connection is normal, right ?
In order to check the root reason, we need to catch module debug log to analyze the OTA message to confirm whether the module have detected the server is closed or it have receive such information, but have no right URC information reported.
Please help to provide your module firmware version with ATI command. I need double check whether it is the software problem. At the same time, please send email to support@quectel.com to get the latest firmware and upgrade it to have a try, also you can ask for the tool to catch debug log. If it still have such issue, you can catch debug log and get local support help you analyze the root reason. Thanks!

[webyfymad]

Hi Kyson,
After discussing with the local FAE Teams, we have come to the conclusion that it is impossible to get a data received acknowledgement from the TCP stack when using SSL.

SEND OK response simply denote the data has been written into the internal TCP send buffer successfully.

AT+QISACK does not work for SSL connections. (works normally for TCP connections)

Alternative recommended to us was to implement our own acknowledgement message (over the already existing communication stack implemented by Quectel)

It is not practical for the server to send an acknowledge for every write to SSL.

Even if such an acknowledge were implemented, it would require waiting for a QSSLURC, and then issuing a AT+QSSLRECV, waiting for the response, and parsing the output for getting the acknowledgement - all this will greatly add latency when we want to get the data out as fast as possible.

Write buffer status should already available at the SSL socket level, and re-implementing acknowledgement with in-band signalling can only be a short term fix. Will this be fixed in a future update?

As of now, my company is considering labeling SSL functionality on MC60 as broken.

Awaiting your reply on this issue as replies from the local teams are quite delayed, probably due the the current pandemic.

Regards.

[Kyson]

Dear Webyfymad,
I have double checked with our RD, it is abnormal that you still can send data but no any URC information while the server have closed the connection. Please catch the module debug log, we need to confirm why the module have not receive such closed information. Thanks!
Please email to support@quectel.com to get the tool to catch debug log. you need to provide the firmware version that you used now. Thanks!

[webyfymad]

Sir,

We have been in contact with the support team. I had uploaded all the necessary logs to them and it is them that confirmed what I had written in the previous comment.

I had mistakenly considered “SEND OK” URC for being a data sent successfully status but it is not.

Normal TCP communication allow checking the number of bytes sent and acknowledged using AT+QISACK .

This does not work for SSL communication.

Certain features have are yet to be implemented in the AT command software.

I wanted to know if there is any plan in the near future to implement a feature to report data sent successfully.

NB: I cannot clear my stack or log the data without knowing if the data has been sent successfully.

[WizIO]

M66/MC60 is based of Mediatek chipset & SDK kernel
This kernel not support SSL confirmation
or if supported it is very difficult to support as AT command

very few kernels/stacks support this confirmation (as Linux, Windows)
but this module is not PC

[webyfymad]

Hello Kyson, can I have a confirmation on what WizIO stated.

As per the catcher logs the internal TCP connection does have ACK messages handled.

[WizIO]

BTW: over ssl / tls the raw data packet IS NOT the tls packet
and test OpenSLL for packet confirmation

[webyfymad]

A status on this issue would be appreciated.

[Kyson]

Dear Webyfymad,
It is difficult to implement what you have said, limit to the Chipset and SDK kernel. And about the issue that the module cannot report the closed URC information need the debug log to double check. We need to confirm whether the module already received such closed information. If you already contact our local FAE, please keep touch with them, and ask them to help analyze the debug log.Thanks!

[asti]

I’m having a similar, but slightly different issue - how do I implement some sort of flow control if AT+QISACK does not work for SSL?

How do I know that the send buffer isn’t overrun. I’m seeing corrupted data at the server during batched writes, so I’m assuming a write buffer overflow.

From what I could find, MediaTek SDK is using xySSL (or mbedTLS) for SSL implementation.
Their SSL API surface seems very limited. But the underlying transport is TCP, and the socket handle should be a valid BSD socket handle as well?

If there’s an AT command which might be a replacement for calling getsockopt, that’s fine too.
Any documentation on what SO_SNDBUF is set to?

tl;dr:
Maybe sync can be implemented at the application layer, but how do I deal with send buffer overruns?

[webyfymad]

Mr Kyson, what @asti brought forward seems to be an issue that I had also experienced randomly.

As for the no closed URC issue, it seems very hard to recreate the exact problem. I am looking into it but still the root issue is the lack of knowledge if the message is sent or not.
At one point I thought of using MQTT but since received message payload is part of a URC, chances of my application missing it is high, and then there is the issue of flow control (as previously mentioned).

[Kyson]

Dear Wedyfymad,
Normally, it is really hard to make sure the SSL connection or TCP connection is infallible, there still will have some problem caused by network, the module also have no method to solve it. In order to make sure the connection is reliable, it is better to check the connection status firstly when you do not send data for long time with AT+QSSLSTATE, of course, even the return value is correct, it still may have the issue like in your test log. So we recommend to send another package of data to double check it. Now, the above is just what module can do it. And about what you said MediaTek SDK for SSL implementation also just can the above.
Normally, your test should be have no issue, we should confirm the existed network issue firstly, then try to solve it. Thanks!