You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 06:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Sun Jul 06, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25993991918,1cyclictest66850irq/125-lan23:40:440
25993991918,1cyclictest66850irq/125-lan23:00:210
25993991918,1cyclictest66850irq/125-lan21:25:160
25993991918,1cyclictest66850irq/125-lan21:25:150
25993991918,1cyclictest66850irq/125-lan21:14:460
25993991818,0cyclictest66850irq/125-lan23:15:140
25993991818,0cyclictest66850irq/125-lan23:05:150
25993991818,0cyclictest66850irq/125-lan22:28:100
25993991818,0cyclictest66850irq/125-lan22:03:040
25993991818,0cyclictest66850irq/125-lan20:05:310
25993991818,0cyclictest66850irq/125-lan00:09:320
25993991817,1cyclictest66850irq/125-lan23:55:180
25993991817,1cyclictest66850irq/125-lan23:52:160
25993991817,1cyclictest66850irq/125-lan23:30:180
25993991817,1cyclictest66850irq/125-lan23:27:050
25993991817,1cyclictest66850irq/125-lan23:22:100
25993991817,1cyclictest66850irq/125-lan22:55:170
25993991817,1cyclictest66850irq/125-lan22:50:200
25993991817,1cyclictest66850irq/125-lan22:45:070
25993991817,1cyclictest66850irq/125-lan22:32:410
25993991817,1cyclictest66850irq/125-lan22:17:050
25993991817,1cyclictest66850irq/125-lan22:05:160
25993991817,1cyclictest66850irq/125-lan21:45:120
25993991817,1cyclictest66850irq/125-lan21:34:130
25993991817,1cyclictest66850irq/125-lan00:20:360
25993991717,0cyclictest66850irq/125-lan21:55:120
25993991717,0cyclictest66850irq/125-lan21:42:010
25993991717,0cyclictest66850irq/125-lan21:36:470
25993991717,0cyclictest66850irq/125-lan21:20:050
25993991717,0cyclictest66850irq/125-lan21:15:130
25993991716,1cyclictest66850irq/125-lan23:45:230
25993991716,1cyclictest66850irq/125-lan23:36:410
25993991716,1cyclictest66850irq/125-lan23:10:070
25993991716,1cyclictest66850irq/125-lan22:40:200
25993991716,1cyclictest66850irq/125-lan22:35:540
25993991716,1cyclictest66850irq/125-lan22:22:540
25993991716,1cyclictest66850irq/125-lan22:13:400
25993991716,1cyclictest66850irq/125-lan21:00:170
25993991716,1cyclictest66850irq/125-lan20:25:230
25993991716,1cyclictest66850irq/125-lan19:13:210
25993991716,1cyclictest66850irq/125-lan00:25:180
25993991716,1cyclictest66850irq/125-lan00:16:560
25993991716,1cyclictest66850irq/125-lan00:12:280
25993991716,1cyclictest66850irq/125-lan00:00:330
25993991616,0cyclictest66850irq/125-lan19:30:240
25993991616,0cyclictest66850irq/125-lan00:35:360
25993991616,0cyclictest66850irq/125-lan00:32:440
25993991615,1cyclictest66850irq/125-lan21:50:140
25993991615,1cyclictest66850irq/125-lan21:05:150
25993991515,0cyclictest66850irq/125-lan20:59:540
25993991515,0cyclictest66850irq/125-lan20:30:240
25993991515,0cyclictest66850irq/125-lan20:20:170
25993991515,0cyclictest66850irq/125-lan19:40:150
25993991515,0cyclictest66850irq/125-lan19:15:210
25993991514,1cyclictest66850irq/125-lan19:50:430
26009991414,0cyclictest771-21snmpd19:45:143
25993991413,1cyclictest66850irq/125-lan20:40:140
25993991413,1cyclictest66850irq/125-lan20:35:130
26009991313,0cyclictest771-21snmpd23:59:283
26009991313,0cyclictest771-21snmpd23:28:543
26009991313,0cyclictest771-21snmpd21:37:293
26009991313,0cyclictest771-21snmpd21:27:203
26009991313,0cyclictest771-21snmpd21:27:203
26009991313,0cyclictest771-21snmpd20:20:363
26009991313,0cyclictest771-21snmpd20:04:593
26000991311,2cyclictest771-21snmpd00:04:371
25993991313,0cyclictest66850irq/125-lan20:10:170
25993991313,0cyclictest66850irq/125-lan20:00:150
25993991313,0cyclictest66850irq/125-lan19:28:220
25993991313,0cyclictest66850irq/125-lan19:20:180
25993991312,1cyclictest66850irq/125-lan20:50:160
26009991212,0cyclictest771-21snmpd23:41:463
26009991212,0cyclictest771-21snmpd23:06:023
26009991212,0cyclictest771-21snmpd19:44:143
26009991212,0cyclictest771-21snmpd00:36:433
26009991212,0cyclictest771-21snmpd00:24:553
26009991212,0cyclictest17293-21cat20:05:153
26009991212,0cyclictest13296-21cat00:15:163
25993991212,0cyclictest66850irq/125-lan20:19:460
25993991212,0cyclictest66850irq/125-lan19:45:210
26009991111,0cyclictest771-21snmpd23:39:243
26009991111,0cyclictest771-21snmpd23:33:563
26009991111,0cyclictest771-21snmpd20:47:493
25993991110,1cyclictest66850irq/125-lan19:59:240
260099998,1cyclictest771-21snmpd22:07:063
260099998,1cyclictest771-21snmpd00:28:553
260049998,1cyclictest771-21snmpd22:28:052
260049998,1cyclictest771-21snmpd20:10:192
260049998,1cyclictest769-21lldpd00:25:252
260049996,3cyclictest771-21snmpd19:58:272
260009998,1cyclictest771-21snmpd22:10:521
260009998,1cyclictest771-21snmpd22:03:361
260009998,1cyclictest771-21snmpd21:36:441
260009998,1cyclictest771-21snmpd20:44:481
259939998,1cyclictest66850irq/125-lan20:45:170
260099988,0cyclictest771-21snmpd23:01:073
260099988,0cyclictest771-21snmpd22:13:043
260099988,0cyclictest771-21snmpd20:33:243
260099988,0cyclictest771-21snmpd00:31:323
260099988,0cyclictest769-21lldpd21:07:383
260099987,1cyclictest771-21snmpd22:50:163
260099987,1cyclictest769-21lldpd19:23:203
260099987,1cyclictest524-21avahi-daemon23:52:073
260099987,1cyclictest524-21avahi-daemon21:47:383
260099987,1cyclictest524-21avahi-daemon21:24:073
260099987,1cyclictest255-20systemd-journal19:10:193
260099986,2cyclictest771-21snmpd22:56:383
260049988,0cyclictest771-21snmpd22:10:132
260049988,0cyclictest771-21snmpd19:38:112
260049988,0cyclictest769-21lldpd23:08:022
260049988,0cyclictest769-21lldpd22:41:382
260049988,0cyclictest769-21lldpd22:18:592
260049988,0cyclictest769-21lldpd00:39:052
260049988,0cyclictest524-21avahi-daemon20:20:262
260049987,1cyclictest771-21snmpd23:32:302
260049987,1cyclictest771-21snmpd22:30:162
260049987,1cyclictest769-21lldpd20:01:192
260049987,1cyclictest524-21avahi-daemon20:51:292
260049987,1cyclictest2918-21ntp_kernel_pll_00:05:162
260049986,2cyclictest771-21snmpd23:11:062
260049986,2cyclictest771-21snmpd22:39:402
260049986,2cyclictest771-21snmpd20:44:552
260009988,0cyclictest771-21snmpd23:55:421
260009988,0cyclictest771-21snmpd23:45:491
260009988,0cyclictest771-21snmpd21:59:491
260009988,0cyclictest769-21lldpd22:53:381
260009988,0cyclictest769-21lldpd22:16:431
260009988,0cyclictest769-21lldpd20:47:301
260009988,0cyclictest769-21lldpd19:36:471
260009987,1cyclictest771-21snmpd19:51:051
259939988,0cyclictest771-21snmpd19:37:100
260099977,0cyclictest771-21snmpd20:17:333
260099977,0cyclictest769-21lldpd22:00:093
260099977,0cyclictest769-21lldpd20:44:353
260099977,0cyclictest769-21lldpd19:18:273
260099976,1cyclictest769-21lldpd22:48:123
260099976,1cyclictest584-21in:imuxsock22:21:103
260099976,1cyclictest524-21avahi-daemon20:50:233
260099976,1cyclictest30699-21unixbench_singl20:35:213
260099976,1cyclictest12645-21fschecks_time19:55:143
260099975,2cyclictest771-21snmpd21:43:593
260099975,2cyclictest749-21ntpd22:16:573
260099975,2cyclictest749-21ntpd00:06:253
260049977,0cyclictest771-21snmpd22:54:552
260049977,0cyclictest771-21snmpd21:42:362
260049977,0cyclictest771-21snmpd20:46:522
260049977,0cyclictest771-21snmpd20:08:202
260049977,0cyclictest769-21lldpd23:50:142
260049977,0cyclictest769-21lldpd23:44:222
260049977,0cyclictest769-21lldpd21:53:072
260049977,0cyclictest769-21lldpd21:34:542
260049977,0cyclictest769-21lldpd20:32:082
260049977,0cyclictest769-21lldpd00:13:392
260049977,0cyclictest517-21systemd-resolve23:46:322
260049976,1cyclictest584-21in:imuxsock00:32:292
260049976,1cyclictest524-21avahi-daemon23:55:032
260049976,1cyclictest524-21avahi-daemon22:21:502
260049976,1cyclictest517-21systemd-resolve21:58:392
260049976,1cyclictest32112-21needreboot22:00:172
260049975,2cyclictest771-21snmpd20:58:462
260049975,2cyclictest771-21snmpd20:18:272
260049973,3cyclictest27957-21awk19:15:002
260009977,0cyclictest771-21snmpd23:24:351
260009977,0cyclictest771-21snmpd20:35:241
260009977,0cyclictest769-21lldpd23:28:301
260009977,0cyclictest769-21lldpd21:10:001
260009977,0cyclictest769-21lldpd20:59:401
260009976,1cyclictest517-21systemd-resolve21:32:231
260009976,1cyclictest2667-21needreboot23:35:161
260009976,1cyclictest14678-21fschecks_time20:00:161
260009975,2cyclictest771-21snmpd21:48:201
260009975,2cyclictest771-21snmpd19:40:361
260009975,2cyclictest771-21snmpd19:29:071
260009974,2cyclictest586-21rs:main1
260099966,0cyclictest769-21lldpd20:29:573
260099966,0cyclictest524-21avahi-daemon22:26:393
260099965,1cyclictest771-21snmpd23:17:113
260099965,1cyclictest769-21lldpd20:13:333
260099965,1cyclictest749-21ntpd23:20:123
260099965,1cyclictest749-21ntpd22:34:423
260099965,1cyclictest749-21ntpd19:50:173
260099965,1cyclictest749-21ntpd19:35:153
260099965,1cyclictest524-21avahi-daemon21:11:483
260099964,2cyclictest771-21snmpd21:19:223
260099964,2cyclictest749-21ntpd21:30:093
260099964,2cyclictest749-21ntpd00:02:223
260099964,2cyclictest13400-21cat23:45:173
260099962,3cyclictest32552-21fw_forwarded_lo19:25:143
260099962,3cyclictest28080-21ssh21:55:503
260049966,0cyclictest771-21snmpd19:42:292
260049966,0cyclictest769-21lldpd00:22:532
260049965,1cyclictest771-21snmpd23:21:112
260049965,1cyclictest771-21snmpd22:58:052
260049965,1cyclictest771-21snmpd19:28:432
260049965,1cyclictest524-21avahi-daemon22:07:362
260049965,1cyclictest32154-21cron21:30:012
260049965,1cyclictest32154-21cron21:30:002
260049964,2cyclictest771-21snmpd19:53:202
260049964,2cyclictest771-21snmpd19:46:382
260049964,1cyclictest586-21rs:main2
260049963,3cyclictest550-21NetworkManager00:15:572
260049963,2cyclictest255-20systemd-journal21:20:132
260049962,4cyclictest550-21NetworkManager21:04:192
260049962,3cyclictest30918-21sshd20:37:342
260049962,3cyclictest25105-21ssh23:25:432
260009966,0cyclictest771-21snmpd21:42:001
260009966,0cyclictest769-21lldpd20:52:021
260009965,1cyclictest771-21snmpd22:58:311
260009965,1cyclictest771-21snmpd20:30:211
260009965,1cyclictest517-21systemd-resolve23:43:191
260009965,1cyclictest20361-21gltestperf22:20:141
260009965,1cyclictest19386-21ntp_kernel_pll_23:50:181
260009964,2cyclictest771-21snmpd21:20:231
260009964,2cyclictest30912-21cut19:20:161
260009964,2cyclictest0-21swapper/123:09:591
260009963,3cyclictest550-21NetworkManager23:16:461
260009963,3cyclictest550-21NetworkManager00:26:101
260009963,3cyclictest1880-21apt-get19:30:151
260009963,2cyclictest31231-21ssh00:33:461
260009963,2cyclictest23157-21ssh21:51:151
260009963,2cyclictest19435-21ssh21:16:281
260009963,2cyclictest1488-21ssh00:35:271
260009962,3cyclictest4722-21sed22:35:221
260009962,3cyclictest12614-21cut21:10:201
260099952,2cyclictest7424-21ssh23:10:043
260099952,2cyclictest23516-21ssh21:51:433
260099952,2cyclictest10681-21ssh22:41:153
260099951,3cyclictest6341-21ssh22:37:363
260099951,3cyclictest255-20systemd-journal21:01:253
260099951,3cyclictest11875-21ssh00:14:173
260049955,0cyclictest769-21lldpd23:18:392
260049955,0cyclictest2253-21fschecks_time23:35:142
260049954,1cyclictest524-21avahi-daemon19:30:502
260049953,2cyclictest771-21snmpd22:47:222
260049953,2cyclictest771-21snmpd21:10:262
260049952,3cyclictest550-21NetworkManager19:23:282
260049952,2cyclictest586-21rs:main2
260049952,2cyclictest31610-21ssh23:00:452
260049952,2cyclictest29766-21ssh00:00:192
260049950,5cyclictest16218-21grep21:45:132
260009955,0cyclictest769-21lldpd23:30:531
260009954,1cyclictest8008-21cpuspeed_turbos19:45:121
260009954,1cyclictest771-21snmpd20:25:151
260009954,1cyclictest769-21lldpd22:46:461
260009954,1cyclictest12454-21cpuspeed_turbos19:55:141
260009953,2cyclictest771-21snmpd23:03:391
260009953,2cyclictest771-21snmpd19:16:341
260009952,2cyclictest9405-21ssh00:10:571
260009952,2cyclictest9282-21grep22:40:171
260009952,2cyclictest6928-21seq22:07:091
260009952,2cyclictest444-21ssh22:31:401
260009952,2cyclictest29027-21ssh21:25:351
260009952,2cyclictest29027-21ssh21:25:341
260009952,2cyclictest26807-21cat19:10:141
260009952,2cyclictest25399-21tr22:25:131
260009952,2cyclictest20601-21ssh00:22:241
260009951,3cyclictest2296-21grep00:05:121
260009951,3cyclictest20028-21sed20:10:241
260009951,3cyclictest12783-21ssh00:15:121
260099941,2cyclictest5739-21tune2fs20:55:153
260099941,2cyclictest2015-21expr19:30:153
260049942,2cyclictest28648-21cat19:15:142
260049941,3cyclictest25641-21fw_forwarded_lo20:25:192
260049940,4cyclictest10222-21ntp_192.168.11421:05:162
260009943,1cyclictest0-21swapper/121:00:141
260009941,3cyclictest550-21NetworkManager20:21:561
260009941,2cyclictest16805-21gpg-connect-age20:05:001
260009940,3cyclictest21166-21fw_packets20:15:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional