netdevsim netdevsim4 netdevsim3 (unregistering): unset [1, 0] type 2 family 0 port 6081 - 0
======================================================
WARNING: possible circular locking dependency detected
syzkaller #0 Tainted: G             L     
------------------------------------------------------
kworker/u8:0/22024 is trying to acquire lock:
ffff8881012ad220 (&root->kernfs_iattr_rwsem){++++}-{4:4}, at: kernfs_unlink_sibling+0x71/0x180

but task is already holding lock:
ffff8881012ad188 (&root->kernfs_rwsem){++++}-{4:4}, at: kernfs_remove_by_name_ns+0x3d/0x130

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #10 (&root->kernfs_rwsem){++++}-{4:4}:
       down_write+0x96/0x200
       kernfs_add_one+0x41/0x5c0
       kernfs_create_dir_ns+0xde/0x130
       internal_create_group+0x425/0x1180
       cpuhp_invoke_callback+0x445/0x860
       cpuhp_issue_call+0x430/0x7a0
       __cpuhp_setup_state_cpuslocked+0x3d9/0x6b0
       __cpuhp_setup_state+0x3f/0x60
       do_one_initcall+0x250/0x8d0
       do_initcall_level+0x104/0x190
       do_initcalls+0x59/0xa0
       kernel_init_freeable+0x2a6/0x3e0
       kernel_init+0x1d/0x1d0
       ret_from_fork+0x51e/0xb90
       ret_from_fork_asm+0x1a/0x30

-> #9 (cpuhp_state_mutex){+.+.}-{4:4}:

-> #8 (cpu_hotplug_lock){++++}-{0:0}:
       cpus_read_lock+0x42/0x160
       static_key_disable+0x12/0x20
       __inet_hash_connect+0x2430/0x2440
       tcp_v4_connect+0xd5f/0x19b0
       __inet_stream_connect+0x25a/0xdd0
       inet_stream_connect+0x66/0xa0
       __sys_connect+0x312/0x450
       __x64_sys_connect+0x7a/0x90
       do_syscall_64+0x14d/0xf80
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #7 (sk_lock-AF_INET){+.+.}-{0:0}:
       lock_sock_nested+0x48/0x100
       inet_shutdown+0x6a/0x390
       nbd_mark_nsock_dead+0x2e9/0x560
       recv_work+0x1c7f/0x1d90
       process_scheduled_works+0xb6e/0x18c0
       worker_thread+0xa53/0xfc0
       kthread+0x388/0x470
       ret_from_fork+0x51e/0xb90
       ret_from_fork_asm+0x1a/0x30

-> #6 (&nsock->tx_lock){+.+.}-{4:4}:
       __mutex_lock+0x19f/0x1300
       nbd_queue_rq+0x37b/0x1100
       blk_mq_dispatch_rq_list+0xa70/0x1910
       __blk_mq_sched_dispatch_requests+0xdcc/0x1600
       blk_mq_sched_dispatch_requests+0xd7/0x190
       blk_mq_run_hw_queue+0x348/0x4f0
       blk_mq_dispatch_list+0xd16/0xe10
       blk_mq_flush_plug_list+0x48d/0x570
       __blk_flush_plug+0x3ed/0x4d0
       __submit_bio+0x28d/0x580
       submit_bio_noacct_nocheck+0x2f4/0xa70
       block_read_full_folio+0x599/0x830
       filemap_read_folio+0x137/0x3b0
       do_read_cache_folio+0x358/0x590
       read_part_sector+0xb6/0x2b0
       adfspart_check_ICS+0xa5/0xa40
       bdev_disk_changed+0x7ba/0x1550
       blkdev_get_whole+0x380/0x510
       bdev_open+0x31e/0xd30
       blkdev_open+0x470/0x610
       do_dentry_open+0x785/0x14e0
       vfs_open+0x3b/0x340
       path_openat+0x2e08/0x3860
       do_file_open+0x23e/0x4a0
       do_sys_openat2+0x113/0x200
       __x64_sys_openat+0x138/0x170
       do_syscall_64+0x14d/0xf80
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #5 (&cmd->lock){+.+.}-{4:4}:
       __mutex_lock+0x19f/0x1300
       nbd_queue_rq+0xc6/0x1100
       blk_mq_dispatch_rq_list+0xa70/0x1910
       __blk_mq_sched_dispatch_requests+0xdcc/0x1600
       blk_mq_sched_dispatch_requests+0xd7/0x190
       blk_mq_run_hw_queue+0x348/0x4f0
       blk_mq_dispatch_list+0xd16/0xe10
       blk_mq_flush_plug_list+0x48d/0x570
       __blk_flush_plug+0x3ed/0x4d0
       __submit_bio+0x28d/0x580
       submit_bio_noacct_nocheck+0x2f4/0xa70
       block_read_full_folio+0x599/0x830
       filemap_read_folio+0x137/0x3b0
       do_read_cache_folio+0x358/0x590
       read_part_sector+0xb6/0x2b0
       adfspart_check_ICS+0xa5/0xa40
       bdev_disk_changed+0x7ba/0x1550
       blkdev_get_whole+0x380/0x510
       bdev_open+0x31e/0xd30
       blkdev_open+0x470/0x610
       do_dentry_open+0x785/0x14e0
       vfs_open+0x3b/0x340
       path_openat+0x2e08/0x3860
       do_file_open+0x23e/0x4a0
       do_sys_openat2+0x113/0x200
       __x64_sys_openat+0x138/0x170
       do_syscall_64+0x14d/0xf80
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #4 (set->srcu){.+.+}-{0:0}:
       __synchronize_srcu+0xca/0x300
       elevator_switch+0x1e8/0x7a0
       elevator_change+0x2cc/0x450
       elevator_set_default+0x36c/0x430
       blk_register_queue+0x366/0x430
       __add_disk+0x677/0xd50
       add_disk_fwnode+0xfb/0x480
       nbd_dev_add+0x72c/0xb50
       nbd_init+0x168/0x1f0
       do_one_initcall+0x250/0x8d0
       do_initcall_level+0x104/0x190
       do_initcalls+0x59/0xa0
       kernel_init_freeable+0x2a6/0x3e0
       kernel_init+0x1d/0x1d0
       ret_from_fork+0x51e/0xb90
       ret_from_fork_asm+0x1a/0x30

-> #3 (&q->elevator_lock){+.+.}-{4:4}:
       __mutex_lock+0x19f/0x1300
       elevator_change+0x1b3/0x450
       elevator_set_none+0xb5/0x140
       blk_mq_update_nr_hw_queues+0x5e7/0x1a60
       nbd_start_device+0x17f/0xb10
       nbd_genl_connect+0x165b/0x1cf0
       genl_family_rcv_msg_doit+0x22a/0x330
       genl_rcv_msg+0x61c/0x7a0
       netlink_rcv_skb+0x232/0x4b0
       genl_rcv+0x28/0x40
       netlink_unicast+0x80f/0x9b0
       netlink_sendmsg+0x813/0xb40
       ____sys_sendmsg+0x972/0x9f0
       ___sys_sendmsg+0x2a5/0x360
       __x64_sys_sendmsg+0x1bd/0x2a0
       do_syscall_64+0x14d/0xf80
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #2 (&q->q_usage_counter(io)#51){++++}-{0:0}:
       blk_alloc_queue+0x546/0x680
       __blk_mq_alloc_disk+0x197/0x390
       nbd_dev_add+0x499/0xb50
       nbd_init+0x168/0x1f0
       do_one_initcall+0x250/0x8d0
       do_initcall_level+0x104/0x190
       do_initcalls+0x59/0xa0
       kernel_init_freeable+0x2a6/0x3e0
       kernel_init+0x1d/0x1d0
       ret_from_fork+0x51e/0xb90
       ret_from_fork_asm+0x1a/0x30

-> #1 (fs_reclaim){+.+.}-{0:0}:
       fs_reclaim_acquire+0x71/0x100
       kmem_cache_alloc_noprof+0x40/0x650
       __kernfs_iattrs+0xdf/0x340
       kernfs_iop_setattr+0xea/0x3f0
       notify_change+0xc1a/0xf40
       do_truncate+0x1c2/0x250
       path_openat+0x2f89/0x3860
       do_file_open+0x23e/0x4a0
       do_sys_openat2+0x113/0x200
       __x64_sys_openat+0x138/0x170
       do_syscall_64+0x14d/0xf80
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&root->kernfs_iattr_rwsem){++++}-{4:4}:
       __lock_acquire+0x15a5/0x2cf0
       lock_acquire+0xf0/0x2e0
       down_write+0x96/0x200
       kernfs_unlink_sibling+0x71/0x180
       __kernfs_remove+0x3e7/0x660
       kernfs_remove_by_name_ns+0xaf/0x130
       sysfs_remove_group+0xfc/0x2e0
       sysfs_remove_groups+0x54/0xb0
       net_rx_queue_update_kobjects+0x68d/0x740
       netdev_unregister_kobject+0x113/0x450
       unregister_netdevice_many_notify+0x1e0e/0x2370
       unregister_netdevice_queue+0x31f/0x360
       nsim_destroy+0x1e5/0x680
       __nsim_dev_port_del+0x14d/0x1b0
       nsim_dev_reload_destroy+0x288/0x490
       nsim_dev_reload_down+0x8a/0xc0
       devlink_reload+0x1d1/0x8d0
       devlink_pernet_pre_exit+0x1ea/0x3f0
       ops_undo_list+0x187/0x940
       cleanup_net+0x56b/0x800
       process_scheduled_works+0xb6e/0x18c0
       worker_thread+0xa53/0xfc0
       kthread+0x388/0x470
       ret_from_fork+0x51e/0xb90
       ret_from_fork_asm+0x1a/0x30

other info that might help us debug this:

Chain exists of:
  &root->kernfs_iattr_rwsem --> cpuhp_state_mutex --> &root->kernfs_rwsem

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&root->kernfs_rwsem);
                               lock(cpuhp_state_mutex);
                               lock(&root->kernfs_rwsem);
  lock(&root->kernfs_iattr_rwsem);

 *** DEADLOCK ***

7 locks held by kworker/u8:0/22024:
 #0: ffff8881012ae148 ((wq_completion)netns){+.+.}-{0:0}, at: process_scheduled_works+0xa52/0x18c0
 #1: ffffc9000673fc40 (net_cleanup_work){+.+.}-{0:0}, at: process_scheduled_works+0xa8d/0x18c0
 #2: ffffffff8fbc1650 (pernet_ops_rwsem){++++}-{4:4}, at: cleanup_net+0xf4/0x800
 #3: ffff88810aa8a130 (&dev->mutex){....}-{4:4}, at: devlink_pernet_pre_exit+0x11b/0x3f0
 #4: ffff88810aa8c250 (&devlink->lock_key#16){+.+.}-{4:4}, at: devlink_pernet_pre_exit+0x12d/0x3f0
 #5: ffffffff8fbcfdc8 (rtnl_mutex){+.+.}-{4:4}, at: nsim_destroy+0xed/0x680
 #6: ffff8881012ad188 (&root->kernfs_rwsem){++++}-{4:4}, at: kernfs_remove_by_name_ns+0x3d/0x130

stack backtrace:
CPU: 0 UID: 0 PID: 22024 Comm: kworker/u8:0 Tainted: G             L      syzkaller #0 PREEMPT(full) 
Tainted: [L]=SOFTLOCKUP
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.2-debian-1.16.2-1 04/01/2014
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 dump_stack_lvl+0xe8/0x150
 print_circular_bug+0x2e1/0x300
 check_noncircular+0x12e/0x150
 __lock_acquire+0x15a5/0x2cf0
 lock_acquire+0xf0/0x2e0
 down_write+0x96/0x200
 kernfs_unlink_sibling+0x71/0x180
 __kernfs_remove+0x3e7/0x660
 kernfs_remove_by_name_ns+0xaf/0x130
 sysfs_remove_group+0xfc/0x2e0
 sysfs_remove_groups+0x54/0xb0
 net_rx_queue_update_kobjects+0x68d/0x740
 netdev_unregister_kobject+0x113/0x450
 unregister_netdevice_many_notify+0x1e0e/0x2370
 unregister_netdevice_queue+0x31f/0x360
 nsim_destroy+0x1e5/0x680
 __nsim_dev_port_del+0x14d/0x1b0
 nsim_dev_reload_destroy+0x288/0x490
 nsim_dev_reload_down+0x8a/0xc0
 devlink_reload+0x1d1/0x8d0
 devlink_pernet_pre_exit+0x1ea/0x3f0
 ops_undo_list+0x187/0x940
 cleanup_net+0x56b/0x800
 process_scheduled_works+0xb6e/0x18c0
 worker_thread+0xa53/0xfc0
 kthread+0x388/0x470
 ret_from_fork+0x51e/0xb90
 ret_from_fork_asm+0x1a/0x30
 </TASK>
netdevsim netdevsim4 netdevsim2 (unregistering): unset [1, 0] type 2 family 0 port 6081 - 0
netdevsim netdevsim4 netdevsim1 (unregistering): unset [1, 0] type 2 family 0 port 6081 - 0
netdevsim netdevsim4 netdevsim0 (unregistering): unset [1, 0] type 2 family 0 port 6081 - 0
bridge_slave_1: left allmulticast mode
bridge_slave_1: left promiscuous mode
bridge0: port 2(bridge_slave_1) entered disabled state
bridge_slave_0: left allmulticast mode
bridge_slave_0: left promiscuous mode
bridge0: port 1(bridge_slave_0) entered disabled state
bond0 (unregistering): (slave bond_slave_0): Releasing backup interface
bond0 (unregistering): (slave bond_slave_1): Releasing backup interface
bond0 (unregistering): Released all slaves
IPVS: stopping backup sync thread 27512 ...
hsr_slave_0: left promiscuous mode
hsr_slave_1: left promiscuous mode
batman_adv: batadv0: Interface deactivated: batadv_slave_0
batman_adv: batadv0: Removing interface: batadv_slave_0
batman_adv: batadv0: Interface deactivated: batadv_slave_1
batman_adv: batadv0: Removing interface: batadv_slave_1
veth1_macvtap: left promiscuous mode
veth0_macvtap: left promiscuous mode
veth1_vlan: left promiscuous mode
veth0_vlan: left promiscuous mode
team0 (unregistering): Port device team_slave_1 removed
team0 (unregistering): Port device team_slave_0 removed
team0 (unregistering): Port device dummy0 removed
netdevsim netdevsim9 eth3 (unregistering): unset [0, 0] type 1 family 0 port 8472 - 0
netdevsim netdevsim9 eth3 (unregistering): unset [1, 0] type 2 family 0 port 20000 - 0
netdevsim netdevsim9 eth3 (unregistering): unset [1, 1] type 2 family 0 port 6081 - 0
netdevsim netdevsim9 eth2 (unregistering): unset [0, 0] type 1 family 0 port 8472 - 0
netdevsim netdevsim9 eth2 (unregistering): unset [1, 0] type 2 family 0 port 20000 - 0
netdevsim netdevsim9 eth2 (unregistering): unset [1, 1] type 2 family 0 port 6081 - 0
netdevsim netdevsim9 eth1 (unregistering): unset [0, 0] type 1 family 0 port 8472 - 0
netdevsim netdevsim9 eth1 (unregistering): unset [1, 0] type 2 family 0 port 20000 - 0
netdevsim netdevsim9 eth1 (unregistering): unset [1, 1] type 2 family 0 port 6081 - 0
netdevsim netdevsim9 eth0 (unregistering): unset [0, 0] type 1 family 0 port 8472 - 0
netdevsim netdevsim9 eth0 (unregistering): unset [1, 0] type 2 family 0 port 20000 - 0
netdevsim netdevsim9 eth0 (unregistering): unset [1, 1] type 2 family 0 port 6081 - 0
bond2 (unregistering): (slave geneve3): Releasing active interface
bond0 (unregistering): left promiscuous mode
bond4 (unregistering): left promiscuous mode
bond0 (unregistering): (slave bond4): Releasing backup interface
bond0 (unregistering): Released all slaves
bond1 (unregistering): Released all slaves
bond2 (unregistering): Released all slaves
bond3 (unregistering): Released all slaves
bond4 (unregistering): Released all slaves
IPVS: stopping master sync thread 25953 ...
hsr_slave_0: left promiscuous mode
hsr_slave_1: left promiscuous mode
veth1_macvtap: left promiscuous mode
veth0_macvtap: left promiscuous mode
veth1_vlan: left allmulticast mode
veth1_vlan: left promiscuous mode
veth0_vlan: left promiscuous mode
batadv0 (unregistering): left allmulticast mode
lo (unregistering): left allmulticast mode
IPVS: stop unused estimator thread 0...
