EG21-G failing to connect to 2degrees in New Zealand

IoT Modules LTE Standard Module
1

Hi there,

I’m working on an embedded Linux product that uses the EG21-G Mini PCIe module. I’m using NetworkManager 1.32.2, ModemManager 1.16.10, and libqmi 1.30.2 to handle the connection for me. I’m using a global Hologram SIM card, with the APN “hologram”. This is working great (as far as I can tell) in the United States, Canada, etc. It also mostly works great in New Zealand, but I’m running into a weird problem:

If the modem happens to select 2degrees as the network to use, it will repeatedly fail to connect. It tries over and over again but never succeeds. It seems to be a QMI error that isn’t documented anywhere that I can find:

[modem0/bearer1] verbose call end reason (3,1078): [cm] (null)

I can’t find anything on this error other than one other person on the libqmi-devel mailing list who never got an answer. It will also print this message sometimes instead:

[modem0/bearer14] verbose call end reason (3,2001): [cm] no-service

If I connect to another network, like Vodafone, it works just fine. I’ve also never reproduced this problem here in the United States where I’ve been developing. I reported this problem to Hologram and they immediately responded telling me that this is something wrong on my end, but I’m suspicious because everything works fine when the modem connects to other networks instead. Anybody have any idea why this is happening? Is there some kind of “EG21-G and 2degrees” incompatibility?

Example connection log:

Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] simple connect started...
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] simple connect state (4/8): wait to get fully enabled
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] simple connect state (5/8): register
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] 3GPP registration state changed (roaming -> registering)
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] 3GPP registration state changed (registering -> home)
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] 3GPP registration state changed (home -> registering)
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] simple connect state (6/8): bearer
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] simple connect state (7/8): connect
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] state changed (registered -> connecting)
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] 3GPP registration state changed (registering -> roaming)
Oct 31 21:49:29 ust20 daemon.info ModemManager[231]: <info>  [modem0] state changed (connecting -> registered)
Oct 31 21:49:29 ust20 daemon.info NetworkManager[239]: <info>  [1667252969.8827] modem["cdc-wdm0"]: modem state changed, 'registered' --> 'connecting' (reason: user-requested)
Oct 31 21:49:29 ust20 daemon.info NetworkManager[239]: <info>  [1667252969.8907] modem["cdc-wdm0"]: modem state changed, 'connecting' --> 'registered' (reason: unknown)
Oct 31 21:49:34 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] couldn't start network: QMI protocol error (14): 'CallFailed'
Oct 31 21:49:34 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] call end reason (1): generic-unspecified
Oct 31 21:49:34 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] verbose call end reason (3,1078): [cm] (null)
Oct 31 21:49:38 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] couldn't start network: QMI protocol error (14): 'CallFailed'
Oct 31 21:49:38 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] call end reason (2): generic-client-end
Oct 31 21:49:38 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] verbose call end reason (3,2000): [cm] client-end
Oct 31 21:49:38 ust20 daemon.warn ModemManager[231]: <warn>  [modem0/bearer1] connection attempt #9 failed: QMI protocol error (14): 'CallFailed'
Oct 31 21:49:38 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer1] connection #9 finished: duration 0s, tx: 0 bytes, rx :0 bytes

And when it retries, the error looks like this instead:

Oct 31 21:49:40 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer19] couldn't start network: QMI protocol error (14): 'CallFailed'
Oct 31 21:49:40 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer19] call end reason (3): generic-no-service
Oct 31 21:49:40 ust20 daemon.info ModemManager[231]: <info>  [modem0/bearer19] verbose call end reason (3,2001): [cm] no-service
Oct 31 21:49:40 ust20 daemon.warn ModemManager[231]: <warn>  [modem0/bearer19] connection attempt #1 failed: QMI protocol error (14): 'CallFailed'

Thanks for any advice!

2

We just bought an actual 2degrees SIM and put it in, and it connects just fine (APN = internet). So it seems like maybe this is limited to just being a Hologram + 2degrees problem. Is there anything in the logs posted above that would help narrow that issue down? Thanks.

3

I used a Hologram SIM for a while here in Australia, in a Quectel BG96 modem, and it worked fine.

I have cancelled that Hologram account, so I can’t test the SIM in one of my broadband Quectel modems.

I have a suspicion that the Hologram SIM will register with an LTE narrowband IoT service (Cat-M1 or NB-IoT), but not on broadband LTE.

That’s RAT 8 or 9 reported by the

AT+COPS=3,2;+COPS?

AT command, and not 7 (broadband).

I also expect that the result of a network scan on your EG21-G using AT+COPS=? will fail to show RAT 8 or 9 being available to that modem.

4

Thanks snowgum!

I don’t have access to the actual modem right this moment because everyone in New Zealand is still sleeping, but I did run AT+COPS? yesterday while it was having trouble connecting and got the following:

+COPS: 0,0,"2degrees Hologram",2

So it looks like it was actually connected on 3G. One thing that’s interesting is that Hologram’s dashboard showed an LTE connection, not 3G. I also did AT+COPS=? and got this:

+COPS: (1,"Spark NZ","Spark NZ","53005",7),(1,"vodafone NZ","voda NZ","53001",7),(1,"2degrees","2degrees","53024",7),(1,"vodafone NZ","voda NZ","53001",2),(1,"vodafone NZ","voda NZ","53001",0),(2,"2degrees","2degrees","53024",2),(1,"Spark NZ","Spark NZ","53005",2),,(0-4),(0-2)

The SIM connects fine with broadband LTE on other providers – typically I see RAT 7 when I’ve got a connection in US/Canada/NZ. Are you thinking that perhaps Hologram only has an agreement for narrowband LTE with 2degrees specifically? I know for sure I got it working with RAT 7 with 2degrees’ SIM. I re-pinged Hologram to see if they have any more detail now that I’ve tested with a 2degrees SIM.

Thanks!

5

It’s registered on 3G. A data connection is a different matter entirely.

Note that 3G network registration does not require an APN (because it has circuit-switched capability), whereas 4G does (being packet domain only).

The ‘2,“2degrees”,“2degrees”,“53024”,2’ confirms 3G registration on network 53024.

The ‘1,“2degrees”,“2degrees”,“53024”,7’ shows 4G signal present without successful network registration.

You could try forcing 4G network registration on 53024 using AT+COPS=1,2,"53024",7

Use AT+COPS=0 to revert to automatic mode.

I was using the “hologram” APN successfully for IoT connections with the BG96 in Australia.

Is this the Hologram SIM in an EG21-G? If so, I don’t have an explanation.

The result from the AT command AT+CGACT? will tell you whether you have a negotiated data connection.

This is for each context defined in your AT+CGDCONT? result. Eg:

AT+CGACT?
+CGACT: 1,1
+CGACT: 2,0
+CGACT: 3,0

Context 1 has an active connection (the “,1”), whereas contexts 2 and 3 don’t.

6

Thanks for clarifying on my incorrect use of the word “connected”! I tried forcing it with the AT+COPS command you listed (although I typically have been using ModemManager through QMI rather than AT commands) and it still seems to fail. Eventually it fell back to One NZ.

Yes. I have the same Hologram SIM in the EG21-G, currently connected to One NZ and working perfectly (I can ping google), and this is what I get:

+COPS: 0,2,"53001",7

So I can definitely get broadband LTE service with a Hologram SIM in New Zealand. I also get broadband LTE service in the US with a Hologram SIM through T-Mobile and AT&T.

When I’m in the failure state registered with 2degrees, I curiously get this for my AT+CGACT status:

AT+CGACT?
+CGACT: 1,1
+CGACT: 2,0
+CGACT: 3,0

Hologram responded to me and they are going to look into it on their end.

7

Thanks for clarifying that your Hologram SIM does work with LTE broadband RAT ID 7.

I take it “One NZ” is another name for “vodafone NZ” as identified for network ID 53001 in your AT+COPS=? result.

Is there a reason you don’t just stick with that network? Insufficient coverage maybe?

Might it be that 2degrees/53024 has no roaming agreement with Hologram (actually Jersey Telecom/23450)?

I expect your Hologram SIM will have this as its home network, with “23450” being the first five digits of the SIM’s IMSI (as shown by the AT+CIMI result).

8

Yeah, I was informed that One NZ is the new rebranded name for Vodafone in New Zealand. We had definitely discussed blocking 2degrees as a potential fix (I’d prefer that over limiting to a single network if possible). I need to research how to accomplish that with ModemManager and/or AT commands.

I would assume that the Hologram SIM wouldn’t try to register on a network that it doesn’t have a roaming agreement for – is my assumption incorrect there? I’ve never noticed any issues with that in the US, but then again I’m also not traveling across the country trying to connect in every little place. I’m just worried that this same kind of problem will happen in other countries such as Australia.

We just discovered that we’re seeing a very similar issue with 2degrees in a different mobile hotspot device that is using a Hologram card. I’m starting to feel more and more confident that this is a Hologram issue and has nothing to do with the EG21-G.

9

You could add 2degrees/53024 to the Hologram SIM’s forbidden PLMN list.

My Hologram SIM’s forbidden list is empty. This is the first entry:

AT+CRSM=176,28539,0,0,3
+CRSM: 144,0,"FFFFFF"
OK

I can overwrite that to block 53024 like so:

AT+CRSM=214,28539,0,0,3,"35F042"

Note the use of “F” as a filler digit to make the MNC F24. With a MCC of 530, the PLMN becomes 530F24 as six digits.

To finish the encoding, the digits are swapped in pairs. Decoding requires “unswapping” the pairs.

Do a hard reset or power-cycle on the modem to reload the forbidden list. The AT+COPS=? command result will then show “53024” entries as forbidden (with a “3”), like this: 3,“2degrees”,“2degrees”,“53024”,2

You can revert to an empty first entry with:

AT+CRSM=214,28539,0,0,3,"FFFFFF"

I would assume this too. The behaviour you’re seeing is odd.

I experienced no such problem.

10

Thank you snowgum! This is great. I had no idea where to even start, and had no idea the forbidden list would be part of the SIM. If Hologram doesn’t fix it, this will definitely be the route I’ll take. Thanks for the detailed explanation!

11

I can’t help but feel you’re using the wrong modem for the Hologram SIM.

That SIM is intended for IoT modems like the Quectel BG96 that I have. The BG96 supports LTE Cat-M1 and NB-IoT, but not broadband.

Nor does the BG96 support 3G/UMTS/WCDMA in any form. It simply isn’t used for IoT.

Your EG21-G supports neither Cat-M1 nor NB-IoT. It’s not an IoT modem.

PS - For further info on the SIM’s forbidden PLMN list see 3GPP 31.102, section 4.2.16.

12

I can understand why you’re thinking that, but Hologram definitely supports broadband LTE on their SIMs as well. For example, their coverage map in New Zealand lists the following support:

Cat-M1, LTE, 3G, 2G

They specifically list Cat-M1 as a separate supported service when it’s supported. In some other countries, e.g. Nicaragua, they support LTE, 3G, and 2G but not Cat-M1.

BTW, I noticed that the modem automatically fills in the forbidden list. My Hologram card here has FirstNet and Sprint listed. Also, the digit ordering is kind of weird – it’s not quite a simple “swap each pair of digits” from what I can tell. For example, FirstNet is 313100 but it shows up in the AT+CRSM output as 130301.

Thanks for the pointer to which 3GPP standard section to look at!

13

I make that 313010, and “Cross Wireless LLC dba Sprocket Wireless” according to https://www.itu.int/dms_pub/itu-t/opb/sp/T-SP-E.212B-2018-PDF-E.pdf

14

I’ve never seen Cross Wireless show up in my network list, but I definitely see FirstNet and it shows up as blocked in my AT+COPS=? output:

'+COPS: (3,"FirstNet","FirstNet","313100",7),(1,"311 588","311 588","311588",7),(1,"U.S.Cellular","USCC","311580",7),(1,"311 589","311 589","311589",7),(2,"AT&T","AT&T","310410",7),(1,"Verizon","Verizon","311480",7),,(0-4),(0-2)'

+CRSM: 144,0,"130301130021FFFFFFFFFFFF"

I would assume the COPS forbidden status output is linked to that same forbidden list, but I could be wrong…

15

You’re quite right. I have been oversimplifying it, and this becomes important with American 6-digit PLMNs.

The encoding algorithm is given in 3GPP 24.008, section 10.5.1.13.

16

Perfect! Thanks for pointing out the exact section in the document. That document’s explanation matches what I determined experimentally…basically the MCC/MNC 123456 is represented in the AT command as 216354. The amount of documentation to go through is super intimidating!

I will check back in with a conclusion as soon as I hear back from Hologram.

17

Agreed. Before encoding, 53024 should be expanded to 53024F.

The encoded value 35F042 remains correct.

1 Like
18

Hologram confirmed on their end that they are seeing issues with 2degrees so they’ve been working with them to fix it. The explanation is way over my head but they said it involves GTP-U. So for some closure on this issue, it definitely isn’t the EG21-G’s fault.