[ACCTON] [SWSS] Failed to start SWSS on init on Accton AS7326-56X, system unable to show interfaces

[mrafatpanah]

Description

I encountered an issue where SWSS fails to start on initialization on the Accton AS7326-56X platform. After upgrading to SONiC 202411, the system does not display interfaces during startup. This issue was not present in the SONiC 202211 version.

Steps to reproduce the issue:

1.Install SONiC 202411 on the Accton AS7326-56X device.
2.Boot the system and observe the interfaces using show interfaces.
3.Check system logs for SWSS-related errors.

Describe the results you received:

SWSS service fails to start during initialization.
Interfaces are not visible when running show interfaces.
Found errors in syslog indicating issues with the initialization of SWSS and Broadcom SDK.

Describe the results you expected:

SWSS should initialize successfully without errors.
Interfaces should be visible and functional after startup.

Output of show version:

SONiC Software Version: SONiC.202411.731834-6b6ac2c8a
SONiC OS Version: 12
Distribution: Debian 12.6
Kernel: 6.1.0-22-2-amd64
Build commit: 6b6ac2c8a
Build date: Fri Dec 27 14:04:21 UTC 2024
Built by: azureuser@3540f203c000004

Platform: x86_64-accton_as7326_56x-r0
HwSKU: Accton-AS7326-56X
ASIC: broadcom
ASIC Count: 1
Serial Number: DS10140307029
Model Number: EGDS10712148U100
Hardware Revision: N/A
Uptime: 10:55:06 up 8 min,  1 user,  load average: 0.73, 0.83, 0.49
Date: Sun 16 Jun 2024 10:55:06

Docker images:
REPOSITORY                    TAG                       IMAGE ID       SIZE
docker-syncd-brcm             202411.731834-6b6ac2c8a   41b05728181d   755MB
docker-syncd-brcm             latest                    41b05728181d   755MB
docker-gbsyncd-broncos        202411.731834-6b6ac2c8a   20226775f2b0   352MB
docker-gbsyncd-broncos        latest                    20226775f2b0   352MB
docker-gbsyncd-credo          202411.731834-6b6ac2c8a   75856c55e3ec   325MB
docker-gbsyncd-credo          latest                    75856c55e3ec   325MB
docker-orchagent              202411.731834-6b6ac2c8a   df2e067e6cd6   356MB
docker-orchagent              latest                    df2e067e6cd6   356MB
docker-fpm-frr                202411.731834-6b6ac2c8a   4617d01f6b6d   377MB
docker-fpm-frr                latest                    4617d01f6b6d   377MB
docker-nat                    202411.731834-6b6ac2c8a   5b4d4b4d9e0f   345MB
docker-nat                    latest                    5b4d4b4d9e0f   345MB
docker-macsec                 latest                    06a4a911d501   345MB
docker-teamd                  202411.731834-6b6ac2c8a   c5330713ef6c   342MB
docker-teamd                  latest                    c5330713ef6c   342MB
docker-sflow                  202411.731834-6b6ac2c8a   ceafc255355f   343MB
docker-sflow                  latest                    ceafc255355f   343MB
docker-dhcp-relay             latest                    4086a3558ac2   323MB
docker-sonic-bmp              202411.731834-6b6ac2c8a   39f95583afd7   314MB
docker-sonic-bmp              latest                    39f95583afd7   314MB
docker-platform-monitor       202411.731834-6b6ac2c8a   7ab72338232f   433MB
docker-platform-monitor       latest                    7ab72338232f   433MB
docker-eventd                 202411.731834-6b6ac2c8a   ec759f316c55   313MB
docker-eventd                 latest                    ec759f316c55   313MB
docker-snmp                   202411.731834-6b6ac2c8a   241d077fe8f4   353MB
docker-snmp                   latest                    241d077fe8f4   353MB
docker-router-advertiser      202411.731834-6b6ac2c8a   e98a7384eab9   313MB
docker-router-advertiser      latest                    e98a7384eab9   313MB
docker-lldp                   202411.731834-6b6ac2c8a   a6735e38652b   359MB
docker-lldp                   latest                    a6735e38652b   359MB
docker-mux                    202411.731834-6b6ac2c8a   5f52c62f2fd3   365MB
docker-mux                    latest                    5f52c62f2fd3   365MB
docker-database               202411.731834-6b6ac2c8a   6362780c53a9   322MB
docker-database               latest                    6362780c53a9   322MB
docker-sonic-gnmi             202411.731834-6b6ac2c8a   8fd04b164f87   403MB
docker-sonic-gnmi             latest                    8fd04b164f87   403MB
docker-sonic-mgmt-framework   202411.731834-6b6ac2c8a   e1b1d116883e   400MB
docker-sonic-mgmt-framework   latest                    e1b1d116883e   400MB

Output of show techsupport:

Additional information you deem important (e.g. issue happens only occasionally):

Syslog problematic part:

2024 Jun 16 10:48:00.535199 sonic DEBUG bgp#bgpcfgd: execute command '['vtysh', '-f', '/tmp/tmpey_qria3']'.
2024 Jun 16 10:48:00.612423 sonic INFO teamd#supervisord 2024-06-16 10:48:00,610 INFO spawned: 'teamsyncd' with pid 25
2024 Jun 16 10:48:00.633592 sonic INFO teamd#supervisord: teamsyncd Listens to teamd events...
2024 Jun 16 10:48:00.689487 sonic NOTICE syncd#syncd: :- threadFunction: time span 1823 ms for 'create:SAI_OBJECT_TYPE_SWITCH:oid:0x21000000000000'
2024 Jun 16 10:48:01.002050 sonic WARNING bgp#mgmtd[36]: [Q5J8K-NY65R] XXX txn in progress, retry init
2024 Jun 16 10:48:01.002050 sonic INFO bgp#mgmtd[36]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:01
2024 Jun 16 10:48:01.003662 sonic ERR syncd#syncd: [none] SAI_API_SWITCH:platform_process_command:430 Platform command "init bcm" failed, rc = -1.
2024 Jun 16 10:48:01.003662 sonic CRIT syncd#syncd: [none] SAI_API_SWITCH:platformInit:1938 initialization command "init bcm" failed, rc = -1 (Internal error).
2024 Jun 16 10:48:01.003701 sonic CRIT syncd#syncd: [none] SAI_API_SWITCH:sai_driver_init:753 Error initializing driver, rc = -1.
2024 Jun 16 10:48:01.003769 sonic CRIT syncd#syncd: [none] SAI_API_SWITCH:brcm_sai_create_switch:1577 initializing SDK failed with error Operation failed (0xfffffff5).
2024 Jun 16 10:48:01.003791 sonic ERR syncd#syncd: :- sendApiResponse: api SAI_COMMON_API_CREATE failed in syncd mode: SAI_STATUS_FAILURE
2024 Jun 16 10:48:01.003936 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_INIT_SWITCH: true
2024 Jun 16 10:48:01.004184 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_FDB_EVENT_NOTIFY: 0x556676020a10
2024 Jun 16 10:48:01.004184 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_PORT_STATE_CHANGE_NOTIFY: 0x556676020a20
2024 Jun 16 10:48:01.004184 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_SWITCH_SHUTDOWN_REQUEST_NOTIFY: 0x556676020a50
2024 Jun 16 10:48:01.004184 sonic ERR syncd#syncd: :- processQuadEvent: attr: SAI_SWITCH_ATTR_SRC_MAC_ADDRESS: 88:2F:64:1F:76:00
2024 Jun 16 10:48:01.004184 sonic ERR swss#orchagent: :- create: create status: SAI_STATUS_FAILURE
2024 Jun 16 10:48:01.004184 sonic ERR swss#orchagent: :- main: Failed to create a switch, rv:-1
2024 Jun 16 10:48:01.191788 sonic NOTICE swss#swssconfig: :- main: Loading config from JSON file:/etc/swss/config.d/ports.json...
2024 Jun 16 10:48:01.441967 sonic INFO swss#supervisord 2024-06-16 10:48:01,441 WARN exited: orchagent (terminated by SIGABRT (core dumped); not expected)
2024 Jun 16 10:48:01.515814 sonic INFO coredump_gen_handler.py[14899]: Global rate_limit_interval period has not passed. Techsupport Invocation is skipped
2024 Jun 16 10:48:01.950903 sonic INFO bgp#supervisord: fpmsyncd Connected!
2024 Jun 16 10:48:01.951503 sonic NOTICE bgp#fpmsyncd: :- setWarmStartState: bgp warm start state changed to disabled
2024 Jun 16 10:48:01.951716 sonic NOTICE bgp#fpmsyncd: :- onRouteMsg: RouteTable del msg for route with only one nh on eth0/docker0: fd00::/80 :: docker0
2024 Jun 16 10:48:02.005342 sonic INFO syncd#supervisord: ledinit polling socket timeout: Success
2024 Jun 16 10:48:02.219202 sonic NOTICE swss#swssconfig: :- main: Loading config from JSON file:/etc/swss/config.d/switch.json...
2024 Jun 16 10:48:02.448185 sonic INFO swss#supervisor-proc-exit-listener: Process 'orchagent' exited unexpectedly. Terminating supervisor 'swss'
2024 Jun 16 10:48:02.448415 sonic NOTICE swss#supervisor-proc-exit-listener: :- publish: EVENT_PUBLISHED: {"sonic-events-host:process-exited-unexpectedly":{"ctr_name":"swss","process_name":"orchagent","timestamp":"2024-06-16T10:48:02.448107Z"}}
2024 Jun 16 10:48:02.449957 sonic INFO swss#supervisord 2024-06-16 10:48:02,449 WARN received SIGTERM indicating exit request

[rahulrock213]

same issue i am getting