Failed to establish connection with RM500Q-GL after reboot on Ubuntu 22.04

We encountered a problem that the 5g connection shows no problem at cold start but the connection could not be established after reboot.

Ubuntu 22.04
Kernel verson
Linux pc-Default-string 6.2.0-39-generic #40~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 16 10:53:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

mmcli version
mmcli 1.20.0
Copyright (2011 - 2022) Aleksander Morgado

Successful log (cold start up)

  -----------------------------------
  General  |                    path: /org/freedesktop/ModemManager1/Modem/0
           |               device id: 4037a5465b911767daf49301d1b7facb8cc6c701
  -----------------------------------
  Hardware |            manufacturer: Quectel
           |                   model: RM500Q-GL
           |       firmware revision: RM500QGLABR13A02M4G
           |          carrier config: ROW_Commercial
           | carrier config revision: 0A010809
           |            h/w revision: 20000
           |               supported: gsm-umts, lte, 5gnr, tds
           |                 current: gsm-umts, lte, 5gnr, tds
           |            equipment id: 863305041810838
  -----------------------------------
  System   |                  device: /sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1
           |                 drivers: option, qmi_wwan
           |                  plugin: quectel
           |            primary port: cdc-wdm1
           |                   ports: cdc-wdm1 (qmi), ttyUSB0 (qcdm), ttyUSB1 (at), ttyUSB2 (at), 
           |                          wwx0a4243c2aaa1 (net)
  -----------------------------------
  Status   |                    lock: sim-pin2
           |          unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (3), sim-puk2 (10)
           |                   state: registered
           |             power state: on
           |             access tech: lte, 5gnr
           |          signal quality: 78% (recent)
  -----------------------------------
  Modes    |               supported: allowed: 3g; preferred: none
           |                          allowed: 4g; preferred: none
           |                          allowed: 3g, 4g; preferred: 4g
           |                          allowed: 3g, 4g; preferred: 3g
           |                          allowed: 5g; preferred: none
           |                          allowed: 4g, 5g; preferred: 5g
           |                          allowed: 4g, 5g; preferred: 4g
           |                          allowed: 3g, 5g; preferred: 5g
           |                          allowed: 3g, 5g; preferred: 3g
           |                          allowed: 3g, 4g, 5g; preferred: 5g
           |                          allowed: 3g, 4g, 5g; preferred: 4g
           |                          allowed: 3g, 4g, 5g; preferred: 3g
           |                 current: allowed: 3g, 4g, 5g; preferred: 5g
  -----------------------------------
  Bands    |               supported: utran-1, utran-3, utran-4, utran-6, utran-5, utran-8, 
           |                          utran-2, eutran-1, eutran-2, eutran-3, eutran-4, eutran-5, eutran-7, 
           |                          eutran-8, eutran-12, eutran-13, eutran-14, eutran-17, eutran-18, 
           |                          eutran-19, eutran-20, eutran-25, eutran-26, eutran-28, eutran-29, 
           |                          eutran-30, eutran-32, eutran-34, eutran-38, eutran-39, eutran-40, 
           |                          eutran-41, eutran-42, eutran-43, eutran-46, eutran-48, eutran-66, 
           |                          eutran-71, utran-19, ngran-1, ngran-2, ngran-3, ngran-5, ngran-7, 
           |                          ngran-8, ngran-12, ngran-20, ngran-25, ngran-28, ngran-38, ngran-40, 
           |                          ngran-41, ngran-48, ngran-66, ngran-71, ngran-77, ngran-78, ngran-79
           |                 current: utran-1, utran-3, utran-4, utran-6, utran-5, utran-8, 
           |                          utran-2, eutran-1, eutran-2, eutran-3, eutran-4, eutran-5, eutran-7, 
           |                          eutran-8, eutran-12, eutran-13, eutran-14, eutran-17, eutran-18, 
           |                          eutran-19, eutran-20, eutran-25, eutran-26, eutran-28, eutran-29, 
           |                          eutran-30, eutran-32, eutran-34, eutran-38, eutran-39, eutran-40, 
           |                          eutran-41, eutran-42, eutran-43, eutran-46, eutran-48, eutran-66, 
           |                          eutran-71, utran-19, ngran-1, ngran-2, ngran-3, ngran-5, ngran-7, 
           |                          ngran-8, ngran-12, ngran-20, ngran-25, ngran-28, ngran-38, ngran-40, 
           |                          ngran-41, ngran-48, ngran-66, ngran-71, ngran-77, ngran-78, ngran-79
  -----------------------------------
  IP       |               supported: ipv4, ipv6, ipv4v6
  -----------------------------------
  3GPP     |                    imei: 863305041810838
           |           enabled locks: fixed-dialing
           |             operator id: 45400
           |           operator name: MORE Mobile
           |            registration: home
           |    packet service state: attached
  -----------------------------------
  3GPP EPS |    ue mode of operation: csps-2
           |     initial bearer path: /org/freedesktop/ModemManager1/Bearer/0
           |  initial bearer ip type: ipv4v6
  -----------------------------------
  SIM      |        primary sim path: /org/freedesktop/ModemManager1/SIM/0
           |          sim slot paths: slot 1: /org/freedesktop/ModemManager1/SIM/0 (active)
           |                          slot 2: none
-- Boot 37efc08fb64e4428ae2b45a667b68334 --
Sep 23 11:22:02 pc-Default-string systemd[1]: Starting Modem Manager...
░░ Subject: A start job for unit ModemManager.service has begun execution
░░ Defined-By: systemd
░░ 
░░ A start job for unit ModemManager.service has begun execution.
░░ 
░░ The job identifier is 167.
Sep 23 11:22:02 pc-Default-string ModemManager[698]: <info>  ModemManager (version 1.20.0) starting in system bus...
Sep 23 11:22:03 pc-Default-string systemd[1]: Started Modem Manager.
░░ Subject: A start job for unit ModemManager.service has finished successfully
░░ Defined-By: systemd
░░ 
░░ A start job for unit ModemManager.service has finished successfully.
░░ 
░░ The job identifier is 167.
Sep 23 11:22:05 pc-Default-string ModemManager[698]: <info>  [device /sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1] creating modem with plugin 'quectel' and '5' ports
Sep 23 11:22:05 pc-Default-string ModemManager[698]: <info>  [base-manager] modem for device '/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1' successfully created
Sep 23 11:22:05 pc-Default-string ModemManager[698]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1c.0/0000:01:00.0': not supported by any plugin
Sep 23 11:22:05 pc-Default-string ModemManager[698]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1c.2/0000:03:00.0': not supported by any plugin
Sep 23 11:22:05 pc-Default-string ModemManager[698]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.4/1-1.4.4': not supported by any plugin
Sep 23 11:22:08 pc-Default-string ModemManager[698]: <warn>  [modem0] unhandled QMI radio interface '9'
Sep 23 11:22:08 pc-Default-string ModemManager[698]: <warn>  [modem0] unhandled QMI radio interface '9'
Sep 23 11:22:08 pc-Default-string ModemManager[698]: <warn>  [modem0/sim0] couldn't load list of emergency numbers: Failed to parse CRSM query result '+CRSM: 148,8,""'
Sep 23 11:22:08 pc-Default-string ModemManager[698]: <warn>  [modem0] couldn't load list of own numbers: Couldn't get MSISDN: QMI protocol error (16): 'NotProvisioned'
Sep 23 11:22:09 pc-Default-string ModemManager[698]: <info>  [modem0] state changed (unknown -> disabled)
Sep 23 11:22:09 pc-Default-string ModemManager[698]: <info>  [modem0] state changed (disabled -> enabling)
Sep 23 11:22:09 pc-Default-string ModemManager[698]: <info>  [modem0] power state updated: on
Sep 23 11:22:10 pc-Default-string ModemManager[698]: <info>  [modem0] state changed (enabling -> enabled)
Sep 23 11:22:10 pc-Default-string ModemManager[698]: <info>  [modem0] 3GPP registration state changed (unknown -> registering)
Sep 23 11:22:10 pc-Default-string ModemManager[698]: <info>  [modem0] 3GPP registration state changed (registering -> home)
Sep 23 11:22:10 pc-Default-string ModemManager[698]: <info>  [modem0] state changed (enabled -> registered)

Failed log (after reboot/init 6)

 --------------------------------
  General  |                 path: /org/freedesktop/ModemManager1/Modem/0
           |            device id: 513c9d8a61c5191985aa8067cf602e7270ca3f71
  --------------------------------
  Hardware |         manufacturer: Quectel
           |                model: RM500Q-GL
           |    firmware revision: RM500QGLABR13A02M4G
           |            supported: gsm-umts, lte
           |              current: gsm-umts, lte
           |         equipment id: 863305041810838
  --------------------------------
  System   |               device: /sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1
           |              drivers: option, qmi_wwan
           |               plugin: quectel
           |         primary port: ttyUSB1
           |                ports: ttyUSB0 (qcdm), ttyUSB1 (at), ttyUSB2 (at), wwan0 (ignored)
  --------------------------------
  Status   |       unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (3), sim-puk2 (10)
           |                state: enabled
           |          power state: on
           |       signal quality: 0% (recent)
  --------------------------------
  Modes    |            supported: allowed: 2g, 3g, 4g; preferred: none
           |              current: allowed: 2g, 3g, 4g; preferred: none
  --------------------------------
  IP       |            supported: ipv4, ipv6, ipv4v6
  --------------------------------
  3GPP     |                 imei: 863305041810838
           |         registration: idle
           | packet service state: detached
  --------------------------------
  3GPP EPS | ue mode of operation: csps-2
  --------------------------------
  SIM      |     primary sim path: /org/freedesktop/ModemManager1/SIM/0
-- Boot ddea21761f764423b8619e89f05e8f9e --
Sep 23 11:25:09 pc-Default-string systemd[1]: Starting Modem Manager...
░░ Subject: A start job for unit ModemManager.service has begun execution
░░ Defined-By: systemd
░░ 
░░ A start job for unit ModemManager.service has begun execution.
░░ 
░░ The job identifier is 173.
Sep 23 11:25:10 pc-Default-string ModemManager[681]: <info>  ModemManager (version 1.20.0) starting in system bus...
Sep 23 11:25:10 pc-Default-string systemd[1]: Started Modem Manager.
░░ Subject: A start job for unit ModemManager.service has finished successfully
░░ Defined-By: systemd
░░ 
░░ A start job for unit ModemManager.service has finished successfully.
░░ 
░░ The job identifier is 173.
Sep 23 11:25:12 pc-Default-string ModemManager[681]: <warn>  [ttyUSB0/probe] failed to parse QCDM version info command result: -7
Sep 23 11:25:13 pc-Default-string ModemManager[681]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1c.0/0000:01:00.0': not supported by any plugin
Sep 23 11:25:13 pc-Default-string ModemManager[681]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1c.2/0000:03:00.0': not supported by any plugin
Sep 23 11:25:13 pc-Default-string ModemManager[681]: <info>  [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.4/1-1.4.4': not supported by any plugin
Sep 23 11:25:57 pc-Default-string ModemManager[681]: <info>  [device /sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1] creating modem with plugin 'quectel' and '5' ports
Sep 23 11:25:57 pc-Default-string ModemManager[681]: <warn>  [plugin/quectel] could not grab port cdc-wdm1: Cannot add port 'usbmisc/cdc-wdm1', unhandled port type
Sep 23 11:25:57 pc-Default-string ModemManager[681]: <info>  [base-manager] modem for device '/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1' successfully created
Sep 23 11:25:58 pc-Default-string ModemManager[681]: <warn>  [modem0/sim0] couldn't load list of emergency numbers: Failed to parse CRSM query result '+CRSM: 148,8,""'
Sep 23 11:25:58 pc-Default-string ModemManager[681]: <warn>  [modem0/sim0] couldn't load GID1: Failed to parse CRSM query result '+CRSM: 103,0,""'
Sep 23 11:25:58 pc-Default-string ModemManager[681]: <warn>  [modem0/sim0] couldn't load GID2: Failed to parse CRSM query result '+CRSM: 103,0,""'
Sep 23 11:25:58 pc-Default-string ModemManager[681]: <info>  [modem0] state changed (unknown -> disabled)
Sep 23 11:25:58 pc-Default-string ModemManager[681]: <info>  [modem0] state changed (disabled -> enabling)
Sep 23 11:25:59 pc-Default-string ModemManager[681]: <info>  [modem0] power state updated: on
Sep 23 11:25:59 pc-Default-string ModemManager[681]: <info>  [modem0] 3GPP registration state changed (unknown -> idle)
Sep 23 11:25:59 pc-Default-string ModemManager[681]: <info>  [modem0] state changed (enabling -> enabled)

Status | unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (3), sim-puk2 (10)
| state: enabled
| power state: on
| signal quality: 0% (recent)

This SIM card doesn’t have a pin code.
It looks like it has not been registered. If possible, you can try to check it using AT commands.

I tried AT commands and it seems the SIM card is already registered.

Quectel
RM500Q-GL
Revision: RM500QGLABR13A02M4G

OK

+CSQ: 28,99

OK

+COPS: 0,0,"MORE Mobile MORE Mobile",13

OK                                                                      
                                                                                                                                                      
+QINISTAT: 7                                                                    
                                                                                
OK                                                                              
                                                                                
+QNWINFO: "FDD LTE","45400","LTE BAND 3",1400                                   
+QNWINFO: "TDD NR5G","45400","NR5G BAND 79",726432                              
                                                                                
OK                                                                              
                                                                                
+QENG: "servingcell","NOCONN"                                                   
+QENG: "LTE","FDD",454,00,545D70C,499,1400,3,5,5,6A3,-95,-12,-62,8,7,160,-      
+QENG: "NR5G-NSA",454,00,809,-106,12,-12,726432,79,6,1                          
                                                                                
OK                                                                              
                                                                                                                                                                                                                             
+CEREG: 0,1                                                                     
                                                                                
OK                                                                              
                                                                                
+CREG: 0,1                                                                      
                                                                                
OK

And I found that /dev/cdc-wdm1 seemed to have some problems after ubuntu reboot.

ModemManager debug log

<debug> [1727159679.490809] [/dev/cdc-wdm1] sent message...
<<<<<< RAW:
<<<<<<   length = 12
<<<<<<   data   = 01:0B:00:00:00:00:00:2D:21:00:00:00...

<debug> [1727159679.500909] [/dev/cdc-wdm1] sent generic request (translated)...
<<<<<< QMUX:
<<<<<<   length  = 11
<<<<<<   flags   = 0x00
<<<<<<   service = "ctl"
<<<<<<   client  = 0
<<<<<< QMI:
<<<<<<   flags       = "none"
<<<<<<   transaction = 45
<<<<<<   tlv_length  = 0
<<<<<<   message     = "Get Version Info" (0x0021)

<debug> [1727159680.468796] [/dev/cdc-wdm1] number of consecutive timeouts: 44
<debug> [1727159680.478978] [/dev/cdc-wdm1] transaction 0x2d aborted, but message is not abortable
<debug> [1727159680.489451] [/dev/cdc-wdm1] sent message...
<<<<<< RAW:
<<<<<<   length = 12
<<<<<<   data   = 01:0B:00:00:00:00:00:2E:21:00:00:00...

<debug> [1727159680.499457] [/dev/cdc-wdm1] sent generic request (translated)...
<<<<<< QMUX:
<<<<<<   length  = 11
<<<<<<   flags   = 0x00
<<<<<<   service = "ctl"
<<<<<<   client  = 0
<<<<<< QMI:
<<<<<<   flags       = "none"
<<<<<<   transaction = 46
<<<<<<   tlv_length  = 0
<<<<<<   message     = "Get Version Info" (0x0021)

<debug> [1727159681.468272] [/dev/cdc-wdm1] number of consecutive timeouts: 45
<debug> [1727159681.478396] [/dev/cdc-wdm1] transaction 0x2e aborted, but message is not abortable
<debug> [1727159681.488280] [cdc-wdm1/qmi] QMI port open operation failed: Transaction timed out
<debug> [1727159681.498381] [cdc-wdm1/probe] error checking QMI support: Transaction timed out
<debug> [1727159681.508161] [cdc-wdm1/probe] port is not QMI-capable
<debug> [1727159681.518059] [plugin-manager] task 0,cdc-wdm1: found best plugin for port (quectel)
<debug> [1727159681.527852] [plugin-manager] task 0,cdc-wdm1: finished in '50.784987' seconds
<debug> [1727159681.538025] [plugin-manager] task 0,cdc-wdm1: best plugin matches device reported one: quectel
<debug> [1727159681.547880] [plugin-manager] task 0: no more ports to probe
<debug> [1727159681.557736] [plugin-manager] task 0: finished in '55.466156' seconds
<info>  [1727159681.567899] [device /sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1] creating modem with plugin 'quectel' and '6' ports
<warn>  [1727159681.580681] [plugin/quectel] could not grab port cdc-wdm1: Cannot add port 'usbmisc/cdc-wdm1', unhandled port type
<debug> [1727159681.590483] [plugin/quectel] ignoring QMI net port wwan0 in non-QMI modem
<debug> [1727159681.600436] [modem0] port 'net/wwan0' grabbed
<debug> [1727159681.610541] [ttyUSB3/at] timeout monitoring enabled in AT port
<debug> [1727159681.620813] [ttyUSB3/at] AT port flagged as secondary
<debug> [1727159681.630987] [modem0] port 'tty/ttyUSB3' grabbed
<debug> [1727159681.640858] [ttyUSB2/at] timeout monitoring enabled in AT port
<debug> [1727159681.650881] [ttyUSB2/at] AT port flagged as primary
<debug> [1727159681.660622] [modem0] port 'tty/ttyUSB2' grabbed
<debug> [1727159681.670367] [modem0] port 'tty/ttyUSB1' grabbed
<debug> [1727159681.680131] [ttyUSB0/qcdm] timeout monitoring enabled in QCDM port
<debug> [1727159681.689872] [modem0] port 'tty/ttyUSB0' grabbed
<debug> [1727159681.700262] [modem0] tty/ttyUSB2 at (primary)
<debug> [1727159681.709968] [modem0] tty/ttyUSB3 at (secondary)
<debug> [1727159681.719611] [modem0] tty/ttyUSB2 at (data primary)
<debug> [1727159681.729063] [modem0] tty/ttyUSB0 qcdm
<debug> [1727159681.738597] [modem0] tty/ttyUSB1 gps (nmea)
<debug> [1727159681.748719] [ttyUSB2/at] opening serial port...
<debug> [1727159681.760401] [ttyUSB2/at] setting up baudrate: 57600
<debug> [1727159681.770383] [ttyUSB2/at] no flow control explicitly requested for device
<debug> [1727159681.780174] [ttyUSB2/at] port attributes not fully set
<debug> [1727159681.789795] [ttyUSB2/at] device open count is 1 (open)
<debug> [1727159681.799313] [ttyUSB2/at] running init sequence...
<info>  [1727159681.808889] [base-manager] modem for device '/sys/devices/pci0000:00/0000:00:1d.0/usb1/1-1/1-1.1' successfully created

Yes. From the log, seems that the /dev/cdc-wdm1 do not work well.

I am wondering why it is wdm1 but not wdm0.

What if you just restarts the modem?

  1. It is wdm0 at first but after after multiple attempts to start/stop ModemManager service, restart modem and reboot machine, it changes to wdm1 somehow and I don’t know why.

  2. It will resume normal if I issue AT command “AT+CFUN=1,1” and then restart ModemManager.

Maybe if the ubuntu reboot, but actually the modem do not restart, and then there is some bugs.
I even guess that there is another proccess that open the cdc-wdm.
You can check
lsof | grep cdc-wdm

Anything special?

qmi-proxy  954                            root    7u      CHR              180,1       0t0        738 /dev/cdc-wdm1
qmi-proxy  954  955 gmain                 root    7u      CHR              180,1       0t0        738 /dev/cdc-wdm1

Seems normal.
qmi-proxy is used by the ModemManager.