RM502Q-AE gets stuck - needs powercycling

Problem:
Every so often, the modem module seems to get stuck. Neither a reset (via mmcli) nor a reboot seems to help. A power cycle is required. (Note: the modem is in an enclosure with external power.) Help?

Background info:

# uname -a
Linux zebra 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023 aarch64 GNU/Linux

# dpkg -l | egrep -i 'qmi|modemmanager'
ii  libqmi-glib5:arm64                   1.26.10-0.1                      arm64        Support library to use the Qualcomm MSM Interface (QMI) protocol
ii  libqmi-proxy                         1.26.10-0.1                      arm64        Proxy to communicate with QMI ports
ii  libqmi-utils                         1.26.10-0.1                      arm64        Utilities to use the QMI protocol from the command line
ii  modemmanager                         1.14.12-0.2                      arm64        D-Bus service for managing modems

# mmcli -m 0
  -----------------------------------
  General  |               dbus path: /org/freedesktop/ModemManager1/Modem/0
           |               device id: 45def29d33a37135e341083a1edb4f94035b7b65
  -----------------------------------
  Hardware |            manufacturer: Quectel
           |                   model: RM502Q-AE
           |       firmware revision: RM502QAEAAR13A03M4G
           |          carrier config: Commercial-TMO
           | carrier config revision: 0A01050F
           |            h/w revision: 20000
           |               supported: gsm-umts, lte, 5gnr
           |                 current: gsm-umts, lte, 5gnr
           |            equipment id: redacted
  -----------------------------------
  System   |                  device: /sys/devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb2/2-1
           |                 drivers: option1, qmi_wwan
           |                  plugin: quectel
           |            primary port: cdc-wdm0
           |                   ports: cdc-wdm0 (qmi), ttyUSB0 (qcdm), ttyUSB1 (at), ttyUSB2 (at),
           |                          wwan0 (net)

Here is everything from syslog when the modem got stuck:

May 22 15:54:22 zebra ModemManager[514]: transaction 0x23dd aborted, but message is not abortable
May 22 15:54:22 zebra ModemManager[514]: <warn>  [modem1/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
May 22 15:54:49 zebra ModemManager[514]: transaction 0x23e4 aborted, but message is not abortable
May 22 15:54:52 zebra ModemManager[514]: transaction 0x23de aborted, but message is not abortable
May 22 15:54:52 zebra ModemManager[514]: <warn>  [modem1/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
May 22 15:55:22 zebra ModemManager[514]: transaction 0x23df aborted, but message is not abortable
May 22 15:55:22 zebra ModemManager[514]: <warn>  [modem1/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
May 22 15:55:24 zebra kernel: [431640.811850] ------------[ cut here ]------------
May 22 15:55:24 zebra kernel: [431640.811908] NETDEV WATCHDOG: wwan0 (qmi_wwan): transmit queue 0 timed out
May 22 15:55:24 zebra kernel: [431640.811986] WARNING: CPU: 2 PID: 0 at net/sched/sch_generic.c:526 dev_watchdog+0x298/0x2a0
May 22 15:55:24 zebra kernel: [431640.812022] Modules linked in: rfcomm cmac algif_hash aes_arm64 aes_generic algif_skcipher af_alg bnep hci_uart btbcm bluetooth ecdh_generic ecc libaes tun nft_chain_nat xt_MASQUERADE nf_nat xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 option nf_defrag_ipv4 xt_HL usb_wwan nft_compat nf_tables qmi_wwan nfnetlink cdc_wdm usbserial brcmfmac brcmutil cfg80211 vc4 v3d snd_soc_hdmi_codec bcm2835_v4l2(C) bcm2835_isp(C) bcm2835_codec(C) drm_display_helper rpivid_hevc(C) cec bcm2835_mmal_vchiq(C) rfkill gpu_sched drm_dma_helper v4l2_mem2mem videobuf2_dma_contig videobuf2_vmalloc drm_kms_helper raspberrypi_hwmon videobuf2_memops drm_shmem_helper snd_soc_core videobuf2_v4l2 videobuf2_common snd_compress snd_bcm2835(C) snd_pcm_dmaengine snd_pcm i2c_brcmstb videodev snd_timer syscopyarea snd vc_sm_cma(C) mc sysfillrect sysimgblt fb_sys_fops nvmem_rmem uio_pdrv_genirq uio drm i2c_dev fuse drm_panel_orientation_quirks backlight ip_tables x_tables ipv6
May 22 15:55:24 zebra kernel: [431640.812435] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G         C         6.1.21-v8+ #1642
May 22 15:55:24 zebra kernel: [431640.812450] Hardware name: Raspberry Pi 4 Model B Rev 1.1 (DT)
May 22 15:55:24 zebra kernel: [431640.812458] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
May 22 15:55:24 zebra kernel: [431640.812471] pc : dev_watchdog+0x298/0x2a0
May 22 15:55:24 zebra kernel: [431640.812484] lr : dev_watchdog+0x298/0x2a0
May 22 15:55:24 zebra kernel: [431640.812494] sp : ffffffc008013df0
May 22 15:55:24 zebra kernel: [431640.812500] x29: ffffffc008013df0 x28: ffffff80fb7ae040 x27: ffffffc008013ee0
May 22 15:55:24 zebra kernel: [431640.812523] x26: ffffffde0ad56000 x25: ffffffde0a3ec880 x24: ffffff804f787488
May 22 15:55:24 zebra kernel: [431640.812544] x23: ffffffde0ad56000 x22: 0000000000000000 x21: ffffff804f7873dc
May 22 15:55:24 zebra kernel: [431640.812564] x20: ffffff804f787000 x19: ffffff804f787488 x18: 0000000000000000
May 22 15:55:24 zebra kernel: [431640.812585] x17: ffffffa2f0d67000 x16: ffffffc008014000 x15: ffffffffffffffff
May 22 15:55:24 zebra kernel: [431640.812605] x14: ffffffde0a8ad858 x13: 74756f2064656d69 x12: ffffffde0ade73b0
May 22 15:55:24 zebra kernel: [431640.812624] x11: 0000000000000003 x10: 0000000000000001 x9 : ffffffde09af054c
May 22 15:55:24 zebra kernel: [431640.812645] x8 : 0000000000017fe8 x7 : c0000000ffffefff x6 : ffffff80fb7ad950
May 22 15:55:24 zebra kernel: [431640.812665] x5 : ffffffa2f0d67000 x4 : 0000000000000040 x3 : 0000000000000104
May 22 15:55:24 zebra kernel: [431640.812685] x2 : 0000000000000103 x1 : 70105334ddd6dd00 x0 : 0000000000000000
May 22 15:55:24 zebra kernel: [431640.812705] Call trace:
May 22 15:55:24 zebra kernel: [431640.812711]  dev_watchdog+0x298/0x2a0
May 22 15:55:24 zebra kernel: [431640.812724]  call_timer_fn+0x38/0x1d0
May 22 15:55:24 zebra kernel: [431640.812741]  run_timer_softirq+0x284/0x520
May 22 15:55:24 zebra kernel: [431640.812756]  __do_softirq+0x1a0/0x4d8
May 22 15:55:24 zebra kernel: [431640.812767]  ____do_softirq+0x18/0x28
May 22 15:55:24 zebra kernel: [431640.812780]  call_on_irq_stack+0x2c/0x54
May 22 15:55:24 zebra kernel: [431640.812792]  do_softirq_own_stack+0x24/0x30
May 22 15:55:24 zebra kernel: [431640.812804]  irq_exit_rcu+0x10c/0x148
May 22 15:55:24 zebra kernel: [431640.812815]  el1_interrupt+0x3c/0x70
May 22 15:55:24 zebra kernel: [431640.812831]  el1h_64_irq_handler+0x18/0x28
May 22 15:55:24 zebra kernel: [431640.812845]  el1h_64_irq+0x64/0x68
May 22 15:55:24 zebra kernel: [431640.812854]  arch_cpu_idle+0x18/0x28
May 22 15:55:24 zebra kernel: [431640.812869]  default_idle_call+0x50/0x188
May 22 15:55:24 zebra kernel: [431640.812881]  do_idle+0x234/0x240
May 22 15:55:24 zebra kernel: [431640.812895]  cpu_startup_entry+0x2c/0x38
May 22 15:55:24 zebra kernel: [431640.812908]  secondary_start_kernel+0x128/0x150
May 22 15:55:24 zebra kernel: [431640.812924]  __secondary_switched+0xb0/0xb4
May 22 15:55:24 zebra kernel: [431640.812942] ---[ end trace 0000000000000000 ]---
May 22 15:55:29 zebra ModemManager[514]: transaction 0x23e5 aborted, but message is not abortable
May 22 15:55:52 zebra ModemManager[514]: transaction 0x23e0 aborted, but message is not abortable
May 22 15:55:52 zebra ModemManager[514]: <warn>  [modem1/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
May 22 15:56:09 zebra ModemManager[514]: transaction 0x23e6 aborted, but message is not abortable
May 22 15:56:20 zebra ModemManager[514]: transaction 0x12 aborted, but message is not abortable
May 22 15:56:22 zebra ModemManager[514]: transaction 0x23e1 aborted, but message is not abortable
May 22 15:56:22 zebra ModemManager[514]: <warn>  [modem1/bearer1] reloading stats failed: QMI operation failed: Transaction timed out

Dear @zdelesde
Which firmware did you use? Did you try the latest firmware?

Yes, I am using RM502QAEAAR13A03M4G.

Note to anyone that may be experiencing the same thing. I switched from QMI to MBIM and the issue has not happened since.

I am also facing the same issue with EC-20 and EC-25 quectel modems, can you explain how you shifted from QMI to MBIM? I am using a raspberry pi based carrier board.

You need to install the mbim drivers and switch the modem to mbim mode using the AT command (search the manual for ‘usbnet’)

1 Like

I am also using the same setup and facing the same issue my device is getting stuck in GSM, and need a manual reset for power. I am using eg25g mini pcie module and raspberry pi pico w as mcu.

is this available on EG25G- mini pcie module.