You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 12:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Wed Nov 19, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12533994443,1cyclictest6597-21fwupd20:13:181
12533991918,1cyclictest66850irq/125-lan23:33:141
12533991918,1cyclictest66850irq/125-lan22:48:221
12533991918,1cyclictest66850irq/125-lan21:23:231
12533991918,1cyclictest66850irq/125-lan00:17:001
12533991818,0cyclictest66850irq/125-lan23:20:341
12533991818,0cyclictest66850irq/125-lan22:55:101
12533991818,0cyclictest66850irq/125-lan22:51:481
12533991818,0cyclictest66850irq/125-lan22:16:101
12533991818,0cyclictest66850irq/125-lan21:49:071
12533991818,0cyclictest66850irq/125-lan21:39:001
12533991818,0cyclictest66850irq/125-lan21:25:061
12533991818,0cyclictest66850irq/125-lan21:19:041
12533991818,0cyclictest66850irq/125-lan00:11:461
12533991817,1cyclictest66850irq/125-lan23:42:061
12533991817,1cyclictest66850irq/125-lan23:38:071
12533991817,1cyclictest66850irq/125-lan23:25:581
12533991817,1cyclictest66850irq/125-lan23:10:461
12533991817,1cyclictest66850irq/125-lan23:06:481
12533991817,1cyclictest66850irq/125-lan22:41:301
12533991817,1cyclictest66850irq/125-lan22:35:161
12533991817,1cyclictest66850irq/125-lan22:34:151
12533991817,1cyclictest66850irq/125-lan22:07:021
12533991817,1cyclictest66850irq/125-lan22:00:241
12533991817,1cyclictest66850irq/125-lan19:59:591
12533991817,1cyclictest66850irq/125-lan00:05:161
12533991717,0cyclictest66850irq/125-lan23:18:581
12533991717,0cyclictest66850irq/125-lan22:20:181
12533991717,0cyclictest66850irq/125-lan22:10:481
12533991717,0cyclictest66850irq/125-lan21:40:101
12533991717,0cyclictest66850irq/125-lan00:20:281
12533991716,1cyclictest66850irq/125-lan23:56:341
12533991716,1cyclictest66850irq/125-lan23:54:041
12533991716,1cyclictest66850irq/125-lan23:48:531
12533991716,1cyclictest66850irq/125-lan23:00:221
12533991716,1cyclictest66850irq/125-lan21:55:071
12533991716,1cyclictest66850irq/125-lan21:50:151
12533991716,1cyclictest66850irq/125-lan21:11:411
12533991716,1cyclictest66850irq/125-lan00:36:321
12533991716,1cyclictest66850irq/125-lan00:28:031
12533991716,1cyclictest66850irq/125-lan00:02:361
12533991616,0cyclictest66850irq/125-lan21:30:111
12533991616,0cyclictest66850irq/125-lan19:40:111
12533991615,1cyclictest66850irq/125-lan20:40:241
12533991615,1cyclictest66850irq/125-lan19:50:151
12533991615,1cyclictest66850irq/125-lan00:33:291
12533991515,0cyclictest66850irq/125-lan22:29:251
12533991515,0cyclictest66850irq/125-lan20:00:201
12533991515,0cyclictest66850irq/125-lan19:20:171
12533991514,1cyclictest66850irq/125-lan21:05:241
12533991514,1cyclictest66850irq/125-lan19:15:241
12533991414,0cyclictest66850irq/125-lan20:51:261
12533991414,0cyclictest66850irq/125-lan20:30:141
12533991413,1cyclictest66850irq/125-lan20:35:161
12533991413,1cyclictest66850irq/125-lan19:45:191
12533991312,1cyclictest66850irq/125-lan20:25:201
12533991312,1cyclictest66850irq/125-lan20:24:351
12533991212,0cyclictest66850irq/125-lan19:29:341
12533991211,1cyclictest66850irq/125-lan20:55:131
12533991211,1cyclictest66850irq/125-lan19:34:551
1253399109,1cyclictest66850irq/125-lan21:00:141
1252599102,5cyclictest15272-21hddtemp_smartct00:35:170
125439998,1cyclictest771-21snmpd20:05:413
125339998,1cyclictest66850irq/125-lan20:46:441
125339998,1cyclictest66850irq/125-lan19:10:151
125259998,1cyclictest771-21snmpd23:33:260
125259998,1cyclictest771-21snmpd19:15:330
125439988,0cyclictest771-21snmpd21:57:433
125439988,0cyclictest771-21snmpd21:42:393
125439988,0cyclictest769-21lldpd23:55:003
125439988,0cyclictest769-21lldpd22:55:133
125439988,0cyclictest769-21lldpd22:34:343
125439988,0cyclictest769-21lldpd21:13:083
125439988,0cyclictest769-21lldpd20:03:283
125439988,0cyclictest769-21lldpd00:38:383
125439988,0cyclictest769-21lldpd00:19:373
125439987,1cyclictest771-21snmpd23:17:563
125439987,1cyclictest771-21snmpd23:09:523
125439987,1cyclictest771-21snmpd21:26:573
125439987,1cyclictest771-21snmpd20:35:183
125439987,1cyclictest771-21snmpd20:31:303
125439987,1cyclictest769-21lldpd22:06:003
125439987,1cyclictest524-21avahi-daemon22:51:263
125439987,1cyclictest524-21avahi-daemon22:15:273
125439987,1cyclictest517-21systemd-resolve00:02:413
125439986,2cyclictest771-21snmpd20:59:513
125439986,2cyclictest771-21snmpd20:50:363
125439986,2cyclictest7312-21cat21:50:173
125389988,0cyclictest771-21snmpd22:37:452
125389988,0cyclictest771-21snmpd22:05:382
125389988,0cyclictest771-21snmpd00:21:222
125389988,0cyclictest769-21lldpd23:07:232
125389988,0cyclictest769-21lldpd22:00:322
125389988,0cyclictest769-21lldpd21:31:512
125389988,0cyclictest769-21lldpd20:06:322
125389988,0cyclictest517-21systemd-resolve23:43:402
125389988,0cyclictest517-21systemd-resolve19:35:452
125389987,1cyclictest771-21snmpd22:18:062
125389987,1cyclictest524-21avahi-daemon21:10:092
125389987,1cyclictest524-21avahi-daemon00:33:472
125389987,1cyclictest524-21avahi-daemon00:09:152
125389987,1cyclictest517-21systemd-resolve00:10:052
125389987,1cyclictest19432-21fschecks_time20:45:152
125389986,2cyclictest771-21snmpd23:32:522
125389986,2cyclictest771-21snmpd23:28:202
125389986,2cyclictest771-21snmpd20:54:212
125339987,1cyclictest66850irq/125-lan20:15:201
125339987,1cyclictest66850irq/125-lan19:35:201
125259988,0cyclictest771-21snmpd23:23:020
125259988,0cyclictest771-21snmpd23:11:010
125259988,0cyclictest769-21lldpd21:35:550
125259987,1cyclictest771-21snmpd20:42:560
125259986,2cyclictest771-21snmpd19:55:420
125259986,2cyclictest749-21ntpd23:02:490
125259986,2cyclictest749-21ntpd21:41:500
125259986,2cyclictest749-21ntpd21:26:240
125259986,2cyclictest749-21ntpd21:16:100
125259986,2cyclictest749-21ntpd21:12:460
125259986,2cyclictest749-21ntpd00:08:480
125439977,0cyclictest771-21snmpd23:29:203
125439977,0cyclictest771-21snmpd23:20:443
125439977,0cyclictest771-21snmpd19:21:333
125439977,0cyclictest769-21lldpd23:11:593
125439977,0cyclictest769-21lldpd22:48:333
125439977,0cyclictest769-21lldpd21:00:493
125439977,0cyclictest769-21lldpd20:47:053
125439976,1cyclictest9264-21cut20:20:153
125439976,1cyclictest771-21snmpd23:59:303
125439976,1cyclictest771-21snmpd22:37:063
125439976,1cyclictest771-21snmpd19:26:363
125439976,1cyclictest771-21snmpd00:20:493
125439976,1cyclictest524-21avahi-daemon20:16:233
125439976,1cyclictest0-21swapper/323:40:153
125439975,2cyclictest771-21snmpd19:53:393
125439975,2cyclictest771-21snmpd00:27:523
125439975,2cyclictest771-21snmpd00:08:543
125389977,0cyclictest771-21snmpd21:44:032
125389977,0cyclictest769-21lldpd23:51:432
125389977,0cyclictest769-21lldpd23:35:372
125389977,0cyclictest769-21lldpd20:38:412
125389977,0cyclictest769-21lldpd19:32:162
125389976,1cyclictest771-21snmpd23:47:012
125389976,1cyclictest524-21avahi-daemon23:01:202
125389976,1cyclictest524-21avahi-daemon00:01:122
125389976,1cyclictest517-21systemd-resolve00:26:552
125389975,2cyclictest771-21snmpd23:13:222
125389975,2cyclictest771-21snmpd21:29:092
125389975,2cyclictest771-21snmpd19:40:092
125389975,2cyclictest17227-21cat19:20:172
125389974,2cyclictest255-20systemd-journal21:39:532
125389973,3cyclictest255-20systemd-journal22:57:462
125389972,4cyclictest6029-21grep21:50:002
125339977,0cyclictest66850irq/125-lan20:05:131
125259977,0cyclictest771-21snmpd23:38:110
125259977,0cyclictest771-21snmpd20:11:580
125259977,0cyclictest771-21snmpd19:11:350
125259977,0cyclictest771-21snmpd00:15:030
125259977,0cyclictest769-21lldpd23:25:110
125259977,0cyclictest769-21lldpd19:51:540
125259977,0cyclictest769-21lldpd00:01:400
125259976,1cyclictest771-21snmpd20:29:490
125259976,1cyclictest771-21snmpd20:01:320
125259976,1cyclictest771-21snmpd19:41:420
125259976,1cyclictest749-21ntpd20:45:020
125259976,1cyclictest21526-21kworker/u16:1-nfsiod21:46:290
125259975,2cyclictest771-21snmpd22:45:380
125259975,2cyclictest771-21snmpd22:00:200
125259975,2cyclictest771-21snmpd20:17:500
125259975,2cyclictest771-21snmpd19:45:130
125259975,2cyclictest749-21ntpd23:07:360
125259975,2cyclictest749-21ntpd21:33:080
125259975,2cyclictest749-21ntpd21:00:020
125259975,2cyclictest749-21ntpd20:50:260
125259975,2cyclictest749-21ntpd19:25:250
125259974,3cyclictest1077-21apt-get22:15:140
125259974,2cyclictest255-20systemd-journal21:54:450
125259974,0cyclictest191rcu_preempt22:52:070
125259973,3cyclictest8329-21ssh23:55:080
125259973,3cyclictest4614-21ssh23:50:460
125259973,3cyclictest12454-21ssh22:55:310
125259973,3cyclictest10614-21ssh21:22:500
125439966,0cyclictest771-21snmpd20:26:503
125439966,0cyclictest771-21snmpd20:12:383
125439966,0cyclictest771-21snmpd19:32:223
125439966,0cyclictest771-21snmpd19:11:333
125439965,1cyclictest771-21snmpd21:34:553
125439965,1cyclictest524-21avahi-daemon19:58:453
125439965,1cyclictest23586-21gltestperf19:35:153
125439965,1cyclictest19918-21cpuspeed_turbos23:35:153
125439964,2cyclictest771-21snmpd21:06:003
125439963,3cyclictest2144-21ssh21:15:133
125439963,3cyclictest1393-21ssh23:48:263
125439963,2cyclictest2794-21ssh21:45:303
125439963,2cyclictest255-20systemd-journal00:12:593
125439962,3cyclictest25648-21ssh21:36:363
125439962,3cyclictest20797-21systemd-run23:05:003
125439962,3cyclictest14823-21ssh22:27:183
125389966,0cyclictest771-21snmpd22:12:462
125389966,0cyclictest771-21snmpd21:15:072
125389966,0cyclictest771-21snmpd20:16:022
125389966,0cyclictest769-21lldpd19:51:522
125389966,0cyclictest769-21lldpd19:26:552
125389965,1cyclictest771-21snmpd20:55:532
125389965,1cyclictest524-21avahi-daemon21:56:042
125389965,1cyclictest524-21avahi-daemon00:19:032
125389965,1cyclictest15654-21ntp_kernel_pll_00:35:192
125389964,2cyclictest771-21snmpd22:22:092
125389964,2cyclictest31992-21cat23:15:182
125389963,3cyclictest550-21NetworkManager23:59:472
125389963,3cyclictest31621-21grep19:55:132
125389963,2cyclictest9666-21ssh21:52:192
125389963,2cyclictest7859-21irqstats21:20:162
125389963,2cyclictest6641-21ld22:50:182
125389963,2cyclictest4837-21ssh22:49:092
125389963,2cyclictest13450-21ssh22:25:242
125389963,2cyclictest13431-21fw_conntrack20:30:152
125389962,3cyclictest32592-21sh22:44:462
125389962,3cyclictest14904-21df_inode19:15:162
125259966,0cyclictest771-21snmpd22:21:030
125259966,0cyclictest0-21swapper/023:46:180
125259965,1cyclictest749-21ntpd23:40:110
125259965,1cyclictest749-21ntpd22:40:370
125259965,1cyclictest749-21ntpd20:05:250
125259965,1cyclictest27352-21cpuspeed_turbos21:05:150
125259965,1cyclictest10287-21cpuspeed_turbos00:30:130
125259965,0cyclictest191rcu_preempt19:35:000
125259963,2cyclictest23916-21dpkg00:10:010
125259963,2cyclictest2101-21ssh23:17:300
125259962,3cyclictest17082-21ssh22:30:100
125259962,3cyclictest14419-21ssh21:56:360
125259960,5cyclictest9429-21ssh00:29:190
125439953,2cyclictest550-21NetworkManager22:21:253
125439952,3cyclictest771-21snmpd20:41:573
125439952,3cyclictest771-21snmpd19:48:583
125439952,3cyclictest31989-21ssh22:13:313
125439952,3cyclictest28804-21ld22:40:183
125439952,2cyclictest27076-21date19:44:593
125439952,2cyclictest20891-21ssh22:03:203
125439952,2cyclictest17014-21ssh23:31:333
125439952,2cyclictest11920-21ssh00:30:523
125439952,2cyclictest10339-21sh21:22:273
125439951,3cyclictest15484-21ntpq19:15:193
125389953,2cyclictest13450-21ntpq19:10:152
125389952,3cyclictest11367-21fw_forwarded_lo20:25:132
125389952,2cyclictest5899-21rm23:20:302
125389952,2cyclictest10191-21timerandwakeup20:20:222
125389950,4cyclictest27477-21forks19:45:162
125389950,4cyclictest1813-21ntpq20:00:222
125259955,0cyclictest769-21lldpd20:55:420
125259954,1cyclictest171ktimers/020:20:140
125259954,1cyclictest171ktimers/000:21:470
125259953,2cyclictest771-21snmpd19:35:130
125259953,2cyclictest550-21NetworkManager22:25:530
125259952,2cyclictest23437-21sed22:35:180
125259952,0cyclictest191rcu_preempt22:10:170
125259950,4cyclictest17238-21irqstats19:20:170
125389941,2cyclictest586-21rs:main2
125389941,2cyclictest5169-21cat20:10:162
125389941,2cyclictest25486-21fschecks_count21:00:142
125389940,3cyclictest27278-21apt-get21:05:152
125389940,3cyclictest17151-21apt-get22:30:142
125259943,1cyclictest0-21swapper/020:35:140
125259943,1cyclictest0-21swapper/020:30:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional