You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 10:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Fri Dec 05, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20495991919,0cyclictest2948-21fwupd00:20:013
20488991918,1cyclictest66850irq/125-lan23:50:111
20488991918,1cyclictest66850irq/125-lan23:08:431
20488991918,1cyclictest66850irq/125-lan22:39:201
20488991918,1cyclictest66850irq/125-lan22:25:191
20488991918,1cyclictest66850irq/125-lan21:35:161
20488991918,1cyclictest66850irq/125-lan21:29:121
20488991918,1cyclictest66850irq/125-lan21:15:141
20488991918,1cyclictest66850irq/125-lan20:25:131
20488991918,1cyclictest66850irq/125-lan00:35:101
20488991918,1cyclictest66850irq/125-lan00:05:171
20488991818,0cyclictest66850irq/125-lan23:58:581
20488991818,0cyclictest66850irq/125-lan21:21:081
20488991818,0cyclictest66850irq/125-lan20:15:121
20488991817,1cyclictest66850irq/125-lan23:42:561
20488991817,1cyclictest66850irq/125-lan23:20:141
20488991817,1cyclictest66850irq/125-lan23:17:421
20488991817,1cyclictest66850irq/125-lan23:10:161
20488991817,1cyclictest66850irq/125-lan22:55:181
20488991817,1cyclictest66850irq/125-lan22:45:201
20488991817,1cyclictest66850irq/125-lan22:08:121
20488991817,1cyclictest66850irq/125-lan21:59:211
20488991817,1cyclictest66850irq/125-lan21:54:071
20488991817,1cyclictest66850irq/125-lan21:32:491
20488991817,1cyclictest66850irq/125-lan00:26:441
20488991817,1cyclictest66850irq/125-lan00:20:201
20488991817,1cyclictest66850irq/125-lan00:15:171
20488991717,0cyclictest66850irq/125-lan23:25:191
20488991717,0cyclictest66850irq/125-lan22:51:291
20488991717,0cyclictest66850irq/125-lan22:30:151
20488991716,1cyclictest66850irq/125-lan23:45:191
20488991716,1cyclictest66850irq/125-lan23:31:051
20488991716,1cyclictest66850irq/125-lan22:42:011
20488991716,1cyclictest66850irq/125-lan22:20:001
20488991716,1cyclictest66850irq/125-lan22:16:081
20488991716,1cyclictest66850irq/125-lan21:47:311
20488991716,1cyclictest66850irq/125-lan21:11:371
20488991716,1cyclictest66850irq/125-lan19:35:121
20488991716,1cyclictest66850irq/125-lan00:30:111
20488991716,1cyclictest66850irq/125-lan00:10:231
20488991716,1cyclictest66850irq/125-lan00:00:151
20488991616,0cyclictest66850irq/125-lan20:14:441
20488991616,0cyclictest66850irq/125-lan20:05:181
20488991615,1cyclictest66850irq/125-lan23:35:311
20488991615,1cyclictest66850irq/125-lan23:02:161
20488991615,1cyclictest66850irq/125-lan22:10:201
20488991615,1cyclictest66850irq/125-lan21:00:001
20488991615,1cyclictest66850irq/125-lan20:20:161
20488991515,0cyclictest66850irq/125-lan22:00:381
20488991515,0cyclictest66850irq/125-lan21:00:241
20488991515,0cyclictest66850irq/125-lan20:35:141
20488991515,0cyclictest66850irq/125-lan20:34:261
20488991515,0cyclictest66850irq/125-lan19:33:591
20488991515,0cyclictest66850irq/125-lan19:25:131
20488991514,1cyclictest66850irq/125-lan21:40:141
20488991514,1cyclictest66850irq/125-lan20:50:011
20488991514,1cyclictest66850irq/125-lan19:50:211
20488991514,1cyclictest66850irq/125-lan19:15:151
20488991414,0cyclictest66850irq/125-lan20:45:111
20488991414,0cyclictest66850irq/125-lan20:40:131
20488991414,0cyclictest66850irq/125-lan19:40:041
20488991413,1cyclictest66850irq/125-lan21:05:251
20488991413,1cyclictest66850irq/125-lan19:20:121
20488991312,1cyclictest66850irq/125-lan19:10:141
20488991211,1cyclictest66850irq/125-lan20:00:171
2048899109,1cyclictest66850irq/125-lan19:55:221
204959998,1cyclictest771-21snmpd23:51:553
204959998,1cyclictest771-21snmpd21:17:533
204959998,1cyclictest771-21snmpd21:00:503
204959998,1cyclictest771-21snmpd19:38:263
204959996,2cyclictest771-21snmpd19:50:063
204929998,1cyclictest771-21snmpd20:24:372
204929998,1cyclictest771-21snmpd19:25:582
204929998,1cyclictest769-21lldpd20:46:132
204889998,1cyclictest66850irq/125-lan19:45:181
204809998,1cyclictest771-21snmpd23:43:040
204809998,1cyclictest771-21snmpd22:54:000
204809998,1cyclictest771-21snmpd22:40:530
204809998,1cyclictest771-21snmpd21:05:050
204959988,0cyclictest771-21snmpd20:15:453
204959988,0cyclictest769-21lldpd23:26:103
204959988,0cyclictest769-21lldpd21:40:443
204959988,0cyclictest769-21lldpd00:15:303
204959988,0cyclictest769-21lldpd00:02:503
204959987,1cyclictest771-21snmpd23:34:283
204959987,1cyclictest771-21snmpd21:25:303
204959987,1cyclictest771-21snmpd19:40:503
204959987,1cyclictest771-21snmpd00:29:433
204959987,1cyclictest517-21systemd-resolve21:21:053
204959987,1cyclictest23352-21grep19:15:153
204959986,2cyclictest771-21snmpd23:01:513
204959986,2cyclictest771-21snmpd20:25:363
204959986,2cyclictest771-21snmpd00:37:393
204959986,2cyclictest18261-21cat20:20:163
204959984,3cyclictest7804-21ntp_states22:45:193
204929988,0cyclictest771-21snmpd23:01:182
204929988,0cyclictest771-21snmpd22:41:452
204929988,0cyclictest771-21snmpd21:56:072
204929988,0cyclictest771-21snmpd21:41:432
204929988,0cyclictest771-21snmpd19:34:472
204929988,0cyclictest769-21lldpd23:49:512
204929988,0cyclictest769-21lldpd23:22:452
204929988,0cyclictest769-21lldpd22:30:162
204929988,0cyclictest769-21lldpd19:13:222
204929987,1cyclictest771-21snmpd21:01:232
204929987,1cyclictest771-21snmpd20:02:362
204929986,2cyclictest771-21snmpd19:41:262
204929985,2cyclictest771-21snmpd00:16:302
204929983,5cyclictest550-21NetworkManager22:08:452
204809988,0cyclictest769-21lldpd22:28:100
204809988,0cyclictest769-21lldpd21:15:470
204809987,1cyclictest771-21snmpd23:49:170
204809987,1cyclictest771-21snmpd23:03:490
204809987,1cyclictest769-21lldpd19:50:400
204809987,1cyclictest769-21lldpd19:18:040
204809987,1cyclictest524-21avahi-daemon23:31:510
204809987,1cyclictest524-21avahi-daemon22:06:070
204809987,1cyclictest524-21avahi-daemon19:48:490
204809986,2cyclictest771-21snmpd23:54:280
204809986,2cyclictest771-21snmpd22:34:330
204809986,2cyclictest771-21snmpd20:28:330
204809986,2cyclictest771-21snmpd00:31:010
204959977,0cyclictest771-21snmpd20:51:023
204959977,0cyclictest771-21snmpd20:42:133
204959977,0cyclictest771-21snmpd19:34:053
204959977,0cyclictest771-21snmpd00:10:033
204959977,0cyclictest769-21lldpd22:14:533
204959977,0cyclictest769-21lldpd21:37:263
204959977,0cyclictest769-21lldpd21:33:503
204959977,0cyclictest517-21systemd-resolve22:28:343
204959976,1cyclictest771-21snmpd20:14:333
204959976,1cyclictest769-21lldpd22:42:503
204959976,1cyclictest524-21avahi-daemon22:06:213
204959976,1cyclictest517-21systemd-resolve00:34:443
204959976,1cyclictest29595-21gltestperf22:35:143
204959976,1cyclictest26886-21cpuspeed_turbos19:25:113
204959976,1cyclictest10865-21latency_hist20:05:003
204959973,3cyclictest23091-21sed20:30:273
204929977,0cyclictest771-21snmpd23:37:462
204929977,0cyclictest771-21snmpd23:10:402
204929977,0cyclictest771-21snmpd22:48:472
204929977,0cyclictest771-21snmpd22:27:032
204929977,0cyclictest771-21snmpd21:05:332
204929977,0cyclictest771-21snmpd20:37:472
204929977,0cyclictest771-21snmpd19:49:292
204929977,0cyclictest769-21lldpd20:51:232
204929977,0cyclictest769-21lldpd20:15:242
204929976,1cyclictest771-21snmpd22:16:492
204929976,1cyclictest771-21snmpd20:42:072
204929976,1cyclictest771-21snmpd19:19:372
204929976,1cyclictest771-21snmpd00:25:192
204929976,1cyclictest769-21lldpd23:54:422
204929976,1cyclictest7477-21cat21:10:142
204929976,1cyclictest410-21cpuspeed_turbos20:55:132
204929976,1cyclictest29585-21gltestperf23:40:132
204929976,1cyclictest16205-21cups-browsed00:00:232
204929975,2cyclictest771-21snmpd21:19:142
204929975,2cyclictest771-21snmpd20:26:032
204929975,2cyclictest771-21snmpd00:32:012
204929973,3cyclictest29078-21ssh22:01:432
204929973,3cyclictest23361-21ssh23:34:122
204809977,0cyclictest771-21snmpd21:50:540
204809977,0cyclictest771-21snmpd20:56:590
204809977,0cyclictest771-21snmpd19:27:460
204809977,0cyclictest769-21lldpd23:19:100
204809977,0cyclictest769-21lldpd20:53:570
204809977,0cyclictest769-21lldpd20:07:140
204809977,0cyclictest517-21systemd-resolve00:10:310
204809976,1cyclictest771-21snmpd20:37:580
204809976,1cyclictest524-21avahi-daemon20:15:150
204809976,1cyclictest524-21avahi-daemon00:08:570
204809976,1cyclictest517-21systemd-resolve22:36:060
204809976,1cyclictest517-21systemd-resolve22:22:570
204809976,1cyclictest10786-21latency_hist20:05:000
204809975,2cyclictest771-21snmpd23:36:580
204809975,2cyclictest771-21snmpd23:23:220
204809975,2cyclictest771-21snmpd23:13:420
204809975,2cyclictest771-21snmpd19:22:070
204809975,2cyclictest771-21snmpd00:21:230
204809975,2cyclictest771-21snmpd00:00:510
204809974,3cyclictest771-21snmpd19:43:560
204809972,5cyclictest550-21NetworkManager00:36:030
204959966,0cyclictest771-21snmpd22:52:413
204959966,0cyclictest771-21snmpd22:15:223
204959966,0cyclictest769-21lldpd19:49:203
204959966,0cyclictest769-21lldpd19:14:303
204959965,1cyclictest524-21avahi-daemon22:02:473
204959964,2cyclictest771-21snmpd21:11:263
204959963,2cyclictest586-21rs:main3
204959963,2cyclictest3898-21ssh23:46:233
204959963,2cyclictest25975-21sed19:20:203
204959963,2cyclictest255-20systemd-journal23:35:503
204959963,2cyclictest14635-21ssh23:57:413
204959962,3cyclictest17159-21ssh22:21:393
204959962,3cyclictest10462-21sed23:20:213
204929966,0cyclictest771-21snmpd22:20:372
204929966,0cyclictest769-21lldpd21:23:122
204929966,0cyclictest517-21systemd-resolve19:58:332
204929965,1cyclictest771-21snmpd22:37:232
204929965,1cyclictest524-21avahi-daemon23:08:262
204929965,1cyclictest12060-21head23:55:172
204929964,2cyclictest771-21snmpd22:57:582
204929963,3cyclictest550-21NetworkManager00:05:452
204929963,2cyclictest586-21rs:main2
204929962,3cyclictest14336-21memory20:10:172
204809966,0cyclictest771-21snmpd23:09:390
204809966,0cyclictest771-21snmpd22:16:170
204809966,0cyclictest771-21snmpd21:36:180
204809966,0cyclictest771-21snmpd20:30:100
204809966,0cyclictest769-21lldpd20:47:150
204809965,1cyclictest771-21snmpd21:03:470
204809965,1cyclictest771-21snmpd00:18:540
204809965,1cyclictest524-21avahi-daemon21:45:170
204809965,1cyclictest524-21avahi-daemon21:26:560
204809965,1cyclictest12039-21gltestperf23:55:160
204809965,0cyclictest771-21snmpd22:03:160
204809964,2cyclictest771-21snmpd19:59:480
204809963,2cyclictest9737-21ssh22:47:310
204809962,3cyclictest8196-21sed00:25:250
204809962,3cyclictest6623-21grep21:40:170
204809962,3cyclictest29357-21ssh21:30:400
204809962,3cyclictest255-20systemd-journal21:55:520
204959955,0cyclictest769-21lldpd23:05:293
204959955,0cyclictest769-21lldpd20:05:293
204959954,1cyclictest11292-21fschecks_count21:45:173
204959952,3cyclictest17271-21fw_packets22:55:163
204959952,3cyclictest1248-21ssh23:10:453
204959952,2cyclictest586-21rs:main3
204959952,2cyclictest5705-21sed23:15:213
204959952,2cyclictest5421-21sed21:05:193
204959952,2cyclictest26255-21ssh22:30:583
204959952,2cyclictest25485-21ssh00:09:393
204959952,2cyclictest21376-21rm21:55:143
204929955,0cyclictest771-21snmpd00:37:302
204929954,1cyclictest787-21fschecks_count21:35:112
204929954,1cyclictest771-21snmpd19:53:182
204929954,1cyclictest769-21lldpd22:10:072
204929953,2cyclictest771-21snmpd19:20:412
204929952,3cyclictest771-21snmpd21:45:392
204929952,3cyclictest771-21snmpd00:21:292
204929952,3cyclictest29012-21sed21:30:262
204929952,2cyclictest26266-21ssh00:10:152
204929952,2cyclictest25148-21ssh21:27:042
204929952,2cyclictest15548-21ssh23:25:212
204929952,2cyclictest13308-21switchtime22:50:212
204929951,3cyclictest550-21NetworkManager19:35:062
204929951,3cyclictest23160-21sed20:30:282
204809955,0cyclictest771-21snmpd23:25:100
204809955,0cyclictest771-21snmpd20:10:360
204809954,1cyclictest771-21snmpd19:10:550
204809953,2cyclictest749-21ntpd19:39:060
204809953,2cyclictest749-21ntpd19:32:200
204809953,2cyclictest550-21NetworkManager21:20:520
204809952,2cyclictest9365-21date22:14:590
204809952,2cyclictest7351-21tail21:10:140
204809951,3cyclictest26207-21fw_forwarded_lo20:40:150
204809951,3cyclictest18640-21sh22:55:460
204809950,5cyclictest0-21swapper/020:20:210
204959943,1cyclictest27997-21cpuspeed_turbos20:45:123
204959941,2cyclictest32524-21dpkg20:55:003
204959941,2cyclictest23986-21grep20:35:143
204959940,4cyclictest7233-21ntpq19:55:183
204929941,2cyclictest12128-21gcc20:05:192
204929940,4cyclictest5203-21ntpq23:15:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional