You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 09:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Nov 08, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25676992019,1cyclictest66850irq/125-lan19:50:022
25676991919,0cyclictest66850irq/125-lan22:34:092
25676991918,1cyclictest66850irq/125-lan23:35:002
25676991918,1cyclictest66850irq/125-lan22:25:272
25676991918,1cyclictest66850irq/125-lan22:10:182
25676991918,1cyclictest66850irq/125-lan21:30:162
25676991918,1cyclictest66850irq/125-lan21:25:192
25676991918,1cyclictest66850irq/125-lan00:25:222
25676991917,1cyclictest66850irq/125-lan23:00:102
25676991818,0cyclictest66850irq/125-lan23:56:422
25676991818,0cyclictest66850irq/125-lan21:45:482
25676991818,0cyclictest66850irq/125-lan00:30:222
25676991817,1cyclictest66850irq/125-lan23:29:182
25676991817,1cyclictest66850irq/125-lan23:12:492
25676991817,1cyclictest66850irq/125-lan23:05:152
25676991817,1cyclictest66850irq/125-lan22:56:572
25676991817,1cyclictest66850irq/125-lan22:40:152
25676991817,1cyclictest66850irq/125-lan22:15:132
25676991817,1cyclictest66850irq/125-lan22:08:142
25676991817,1cyclictest66850irq/125-lan21:37:122
25676991817,1cyclictest66850irq/125-lan21:20:132
25676991817,1cyclictest66850irq/125-lan21:14:072
25676991817,1cyclictest66850irq/125-lan00:22:202
25676991817,1cyclictest66850irq/125-lan00:15:182
25676991717,0cyclictest66850irq/125-lan23:41:292
25676991717,0cyclictest66850irq/125-lan23:35:172
25676991717,0cyclictest66850irq/125-lan22:50:132
25676991717,0cyclictest66850irq/125-lan19:45:172
25676991717,0cyclictest66850irq/125-lan00:12:532
25676991716,1cyclictest66850irq/125-lan23:45:102
25676991716,1cyclictest66850irq/125-lan23:20:132
25676991716,1cyclictest66850irq/125-lan23:18:332
25676991716,1cyclictest66850irq/125-lan22:45:262
25676991716,1cyclictest66850irq/125-lan22:35:152
25676991716,1cyclictest66850irq/125-lan22:20:152
25676991716,1cyclictest66850irq/125-lan22:00:142
25676991716,1cyclictest66850irq/125-lan21:40:322
25676991716,1cyclictest66850irq/125-lan21:18:132
25676991716,1cyclictest66850irq/125-lan20:35:202
25676991716,1cyclictest66850irq/125-lan20:10:172
25676991716,1cyclictest66850irq/125-lan20:05:132
25676991716,1cyclictest66850irq/125-lan19:55:182
25676991716,1cyclictest66850irq/125-lan19:35:192
25676991716,1cyclictest66850irq/125-lan19:30:122
25676991716,1cyclictest66850irq/125-lan00:35:122
25676991716,1cyclictest66850irq/125-lan00:05:452
25676991616,0cyclictest66850irq/125-lan20:15:212
25676991615,1cyclictest66850irq/125-lan23:50:482
25676991615,1cyclictest66850irq/125-lan21:56:232
25676991615,1cyclictest66850irq/125-lan21:50:182
25676991615,1cyclictest66850irq/125-lan20:30:192
25676991615,1cyclictest66850irq/125-lan20:00:162
25676991515,0cyclictest66850irq/125-lan20:58:262
25676991514,1cyclictest66850irq/125-lan20:50:132
25676991514,1cyclictest66850irq/125-lan20:40:152
25676991514,1cyclictest66850irq/125-lan00:00:252
25676991413,1cyclictest66850irq/125-lan21:05:142
25676991313,0cyclictest66850irq/125-lan20:25:262
25676991313,0cyclictest66850irq/125-lan19:10:152
25676991312,1cyclictest66850irq/125-lan20:45:212
25676991312,1cyclictest66850irq/125-lan19:22:262
25676991212,0cyclictest66850irq/125-lan19:25:142
25676991110,1cyclictest66850irq/125-lan20:20:202
25671991111,0cyclictest5421-21fwupd23:45:131
2567699109,1cyclictest66850irq/125-lan21:00:182
25676991010,0cyclictest66850irq/125-lan19:15:172
256799996,3cyclictest771-21snmpd23:09:313
256799996,3cyclictest771-21snmpd20:07:593
256719998,1cyclictest771-21snmpd22:43:061
256719998,1cyclictest541-21systemd-logind21:20:211
256649998,1cyclictest771-21snmpd23:49:380
256649998,1cyclictest771-21snmpd20:38:050
256799988,0cyclictest771-21snmpd23:21:403
256799988,0cyclictest771-21snmpd19:57:463
256799988,0cyclictest771-21snmpd19:27:513
256799988,0cyclictest769-21lldpd23:11:193
256799988,0cyclictest769-21lldpd20:03:113
256799987,1cyclictest771-21snmpd22:39:543
256799987,1cyclictest771-21snmpd22:30:573
256799987,1cyclictest771-21snmpd22:20:023
256799987,1cyclictest769-21lldpd21:38:323
256799987,1cyclictest517-21systemd-resolve22:19:363
256799986,2cyclictest771-21snmpd23:52:023
256799986,2cyclictest771-21snmpd23:48:113
256799986,2cyclictest771-21snmpd20:43:183
256799986,2cyclictest771-21snmpd00:39:013
256719988,0cyclictest771-21snmpd21:15:241
256719988,0cyclictest771-21snmpd19:18:001
256719988,0cyclictest769-21lldpd23:15:091
256719988,0cyclictest769-21lldpd22:46:431
256719988,0cyclictest769-21lldpd20:20:401
256719988,0cyclictest769-21lldpd19:27:261
256719987,1cyclictest771-21snmpd22:37:571
256719987,1cyclictest771-21snmpd22:32:241
256719987,1cyclictest771-21snmpd21:52:371
256719987,1cyclictest771-21snmpd20:29:381
256719987,1cyclictest769-21lldpd23:32:521
256719987,1cyclictest769-21lldpd19:39:191
256719986,2cyclictest771-21snmpd22:02:141
256719986,2cyclictest771-21snmpd19:12:421
256719986,2cyclictest749-21ntpd19:30:531
256649988,0cyclictest771-21snmpd23:10:490
256649988,0cyclictest771-21snmpd20:58:510
256649988,0cyclictest769-21lldpd23:41:180
256649988,0cyclictest769-21lldpd23:00:170
256649988,0cyclictest769-21lldpd20:27:390
256649988,0cyclictest769-21lldpd19:42:540
256649987,1cyclictest771-21snmpd23:39:020
256649987,1cyclictest771-21snmpd22:26:550
256649987,1cyclictest771-21snmpd21:46:430
256649987,1cyclictest771-21snmpd20:50:570
256649987,1cyclictest771-21snmpd19:26:180
256649987,1cyclictest771-21snmpd00:13:090
256649987,1cyclictest769-21lldpd20:48:130
256649987,1cyclictest524-21avahi-daemon00:28:130
256649986,2cyclictest771-21snmpd23:27:130
256649986,2cyclictest771-21snmpd19:12:210
256649986,2cyclictest771-21snmpd00:01:480
256649985,3cyclictest771-21snmpd22:06:590
256799977,0cyclictest771-21snmpd23:16:073
256799977,0cyclictest771-21snmpd20:51:393
256799977,0cyclictest769-21lldpd20:25:093
256799977,0cyclictest769-21lldpd20:23:193
256799977,0cyclictest524-21avahi-daemon20:15:103
256799976,1cyclictest771-21snmpd23:25:343
256799976,1cyclictest771-21snmpd22:14:383
256799976,1cyclictest771-21snmpd21:56:383
256799976,1cyclictest769-21lldpd19:20:043
256799976,1cyclictest524-21avahi-daemon23:36:473
256799976,1cyclictest524-21avahi-daemon19:35:173
256799976,1cyclictest517-21systemd-resolve00:06:193
256799976,1cyclictest26131-21fschecks_time19:10:153
256799976,1cyclictest10289-21fschecks_time19:50:143
256799975,2cyclictest771-21snmpd21:24:303
256799975,2cyclictest771-21snmpd21:02:333
256799972,4cyclictest255-20systemd-journal21:12:433
256769977,0cyclictest771-21snmpd19:43:552
256719977,0cyclictest771-21snmpd23:20:131
256719977,0cyclictest771-21snmpd00:38:011
256719977,0cyclictest769-21lldpd22:07:301
256719977,0cyclictest769-21lldpd20:54:471
256719977,0cyclictest769-21lldpd19:40:441
256719977,0cyclictest769-21lldpd00:03:361
256719976,1cyclictest771-21snmpd20:00:341
256719976,1cyclictest769-21lldpd19:53:361
256719976,1cyclictest517-21systemd-resolve21:41:491
256719976,1cyclictest287-21systemd-udevd23:38:211
256719976,1cyclictest255-20systemd-journal20:30:141
256719975,2cyclictest771-21snmpd21:58:021
256719975,2cyclictest771-21snmpd00:18:271
256719975,2cyclictest771-21snmpd00:06:001
256649977,0cyclictest771-21snmpd23:16:250
256649977,0cyclictest771-21snmpd21:28:400
256649977,0cyclictest771-21snmpd19:32:010
256649977,0cyclictest771-21snmpd19:20:140
256649977,0cyclictest769-21lldpd19:56:230
256649976,1cyclictest771-21snmpd22:50:150
256649976,1cyclictest769-21lldpd21:32:580
256649976,1cyclictest524-21avahi-daemon21:50:090
256649976,1cyclictest524-21avahi-daemon21:20:230
256649976,1cyclictest11465-21fschecks_count22:45:170
256649975,2cyclictest771-21snmpd21:04:060
256649975,2cyclictest771-21snmpd20:30:080
256799966,0cyclictest771-21snmpd23:57:563
256799966,0cyclictest769-21lldpd19:42:343
256799965,1cyclictest771-21snmpd22:45:333
256799965,1cyclictest524-21avahi-daemon00:20:373
256799965,1cyclictest32655-21gltestperf23:40:133
256799965,1cyclictest30361-21cat00:10:153
256799964,2cyclictest771-21snmpd21:51:493
256799964,2cyclictest771-21snmpd20:46:093
256799964,2cyclictest771-21snmpd00:27:523
256799964,2cyclictest28633-21smartctl20:35:173
256799963,3cyclictest550-21NetworkManager21:42:053
256799963,2cyclictest586-21rs:main3
256799963,2cyclictest28181-21ssh23:01:323
256799963,2cyclictest255-20systemd-journal21:32:053
256799962,3cyclictest5385-21ssh00:17:243
256799962,3cyclictest255-20systemd-journal22:54:183
256799962,3cyclictest25309-21ssh22:59:293
256799961,2cyclictest9367-21sh19:50:003
256719966,0cyclictest771-21snmpd20:48:451
256719966,0cyclictest769-21lldpd20:19:571
256719966,0cyclictest769-21lldpd20:06:021
256719966,0cyclictest769-21lldpd19:46:081
256719966,0cyclictest524-21avahi-daemon21:02:451
256719965,1cyclictest771-21snmpd21:08:031
256719965,1cyclictest541-21systemd-logind23:06:211
256719965,1cyclictest287-21systemd-udevd20:38:201
256719965,1cyclictest255-20systemd-journal22:17:381
256719965,1cyclictest16284-21cpuspeed_turbos22:50:131
256719964,2cyclictest771-21snmpd23:55:321
256719964,2cyclictest771-21snmpd22:59:201
256719964,2cyclictest771-21snmpd19:23:081
256719963,3cyclictest5194-21sshd23:11:261
256719963,2cyclictest749-21ntpd23:00:141
256719963,2cyclictest5677-21idleruntime-cro21:00:011
256719962,4cyclictest550-21NetworkManager23:28:431
256719962,3cyclictest28075-21ssh21:26:531
256719962,3cyclictest19987-21ssh00:33:371
256719961,5cyclictest550-21NetworkManager23:41:571
256649966,0cyclictest771-21snmpd21:07:120
256649966,0cyclictest771-21snmpd20:22:230
256649966,0cyclictest769-21lldpd00:09:120
256649965,1cyclictest771-21snmpd22:33:390
256649965,1cyclictest524-21avahi-daemon23:54:110
256649965,1cyclictest18431-21gltestperf22:20:140
256649964,2cyclictest771-21snmpd23:34:350
256649964,2cyclictest771-21snmpd22:15:230
256649964,2cyclictest771-21snmpd22:00:410
256649963,3cyclictest550-21NetworkManager21:12:190
256649963,2cyclictest586-21rs:main0
256649963,2cyclictest3791-21ssh22:36:480
256649963,2cyclictest19087-21ssh21:18:490
256649962,3cyclictest7694-21fw_conntrack00:20:140
256799955,0cyclictest771-21snmpd22:41:453
256799955,0cyclictest769-21lldpd20:12:113
256799954,1cyclictest8812-21ntpq21:05:143
256799954,1cyclictest771-21snmpd21:25:073
256799954,1cyclictest771-21snmpd19:15:453
256799954,1cyclictest14139-21fschecks_time21:45:113
256799953,2cyclictest771-21snmpd20:55:483
256799953,2cyclictest771-21snmpd19:31:213
256799953,2cyclictest550-21NetworkManager00:00:233
256799952,3cyclictest749-21ntpd22:00:073
256799952,3cyclictest550-21NetworkManager00:30:053
256799952,3cyclictest15649-21if_lan21:15:153
256799952,2cyclictest22797-21sh23:30:143
256799951,3cyclictest25475-21ssh22:26:303
256719954,1cyclictest0-21swapper/122:25:181
256719953,2cyclictest771-21snmpd21:30:101
256719953,2cyclictest550-21NetworkManager23:51:571
256719952,3cyclictest18425-21fw_forwarded_lo20:10:131
256719952,2cyclictest8934-21ssh00:20:321
256719952,2cyclictest64-21kcompactd021:10:191
256719952,2cyclictest4759-21wc21:35:231
256719952,2cyclictest14891-21wc21:45:161
256719950,4cyclictest12994-21ntpq00:25:181
256649955,0cyclictest771-21snmpd19:36:180
256649953,2cyclictest771-21snmpd22:55:240
256649953,2cyclictest771-21snmpd20:12:110
256649953,2cyclictest550-21NetworkManager22:42:590
256649953,2cyclictest16604-21ssh23:24:230
256649952,2cyclictest5052-21ssh00:16:540
256649952,2cyclictest30559-21fw_conntrack20:40:140
256649952,2cyclictest22325-21sed00:35:200
256649952,2cyclictest18524-21ssh00:31:000
256649952,2cyclictest12162-21ssh21:43:080
256649951,3cyclictest17206-21sed20:05:220
256649951,3cyclictest15262-21sed23:55:170
256649950,4cyclictest25536-21ntpq21:55:190
256799942,2cyclictest255-20systemd-journal20:30:063
256719942,2cyclictest771-21snmpd22:10:081
256719940,3cyclictest30437-21diskstats20:40:131
256649942,2cyclictest771-21snmpd20:15:320
256649942,0cyclictest0-21swapper/019:15:120
256649941,2cyclictest8280-21tail19:45:150
256649941,2cyclictest14252-21grep20:00:130
256649940,3cyclictest7763-21ssh22:10:060
256649940,3cyclictest10618-21/usr/sbin/munin19:50:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional