You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 12:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Wed Nov 26, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3048991918,1cyclictest66850irq/125-lan23:58:253
3048991918,1cyclictest66850irq/125-lan23:50:163
3048991918,1cyclictest66850irq/125-lan23:40:183
3048991918,1cyclictest66850irq/125-lan22:15:163
3048991918,1cyclictest66850irq/125-lan00:08:563
3048991818,0cyclictest66850irq/125-lan23:14:293
3048991818,0cyclictest66850irq/125-lan21:13:003
3048991817,1cyclictest66850irq/125-lan23:30:133
3048991817,1cyclictest66850irq/125-lan23:24:023
3048991817,1cyclictest66850irq/125-lan22:46:493
3048991817,1cyclictest66850irq/125-lan22:42:253
3048991817,1cyclictest66850irq/125-lan22:36:313
3048991817,1cyclictest66850irq/125-lan22:33:383
3048991817,1cyclictest66850irq/125-lan22:25:003
3048991817,1cyclictest66850irq/125-lan22:10:173
3048991817,1cyclictest66850irq/125-lan22:00:093
3048991817,1cyclictest66850irq/125-lan21:46:363
3048991817,1cyclictest66850irq/125-lan21:35:333
3048991817,1cyclictest66850irq/125-lan21:32:573
3048991817,1cyclictest66850irq/125-lan21:25:143
3048991817,1cyclictest66850irq/125-lan21:19:543
3048991817,1cyclictest66850irq/125-lan20:25:153
3048991817,1cyclictest66850irq/125-lan00:36:383
3048991817,1cyclictest66850irq/125-lan00:32:543
3048991817,1cyclictest66850irq/125-lan00:24:393
3048991817,1cyclictest66850irq/125-lan00:15:203
3048991817,1cyclictest66850irq/125-lan00:10:213
3048991717,0cyclictest66850irq/125-lan23:25:243
3048991717,0cyclictest66850irq/125-lan23:15:183
3048991717,0cyclictest66850irq/125-lan20:08:533
3048991716,1cyclictest66850irq/125-lan23:05:193
3048991716,1cyclictest66850irq/125-lan22:55:413
3048991716,1cyclictest66850irq/125-lan22:53:573
3048991716,1cyclictest66850irq/125-lan21:55:163
3048991716,1cyclictest66850irq/125-lan21:51:003
3048991716,1cyclictest66850irq/125-lan21:40:163
3048991716,1cyclictest66850irq/125-lan20:10:193
3048991716,1cyclictest66850irq/125-lan19:10:153
3048991616,0cyclictest66850irq/125-lan21:20:293
3048991616,0cyclictest66850irq/125-lan20:03:413
3048991615,1cyclictest66850irq/125-lan23:45:533
3048991615,1cyclictest66850irq/125-lan23:36:003
3048991615,1cyclictest66850irq/125-lan23:00:053
3048991615,1cyclictest66850irq/125-lan22:21:063
3048991615,1cyclictest66850irq/125-lan22:07:383
3048991615,1cyclictest66850irq/125-lan00:25:263
3048991615,1cyclictest66850irq/125-lan00:00:173
3048991515,0cyclictest66850irq/125-lan20:55:173
3048991515,0cyclictest66850irq/125-lan20:52:093
3048991515,0cyclictest66850irq/125-lan19:45:153
3048991514,1cyclictest66850irq/125-lan21:00:143
3048991514,1cyclictest66850irq/125-lan19:50:133
3048991514,1cyclictest66850irq/125-lan19:40:143
3048991414,0cyclictest66850irq/125-lan21:05:113
3048991413,1cyclictest66850irq/125-lan19:30:433
3048991413,1cyclictest66850irq/125-lan19:15:193
3048991313,0cyclictest66850irq/125-lan20:30:123
3048991312,1cyclictest66850irq/125-lan20:22:403
3048991312,1cyclictest66850irq/125-lan19:35:233
3048991211,1cyclictest66850irq/125-lan20:40:133
3048991211,1cyclictest66850irq/125-lan19:55:173
3048991111,0cyclictest66850irq/125-lan20:15:163
3048991111,0cyclictest66850irq/125-lan19:22:293
303699116,5cyclictest771-21snmpd19:25:520
303699116,4cyclictest771-21snmpd20:40:000
303699110,3cyclictest24449-21grep20:00:220
303699107,3cyclictest769-21lldpd20:40:260
303699100,3cyclictest23812-21ssh23:26:020
30449998,1cyclictest769-21lldpd22:42:042
30449998,1cyclictest769-21lldpd21:54:522
30399998,1cyclictest769-21lldpd21:00:541
30369997,2cyclictest771-21snmpd22:13:210
30369996,3cyclictest771-21snmpd21:26:040
30369995,2cyclictest749-21ntpd21:08:160
30369994,5cyclictest21809-21cat19:55:140
30369991,3cyclictest550-21NetworkManager22:36:180
30369990,5cyclictest0-21swapper/021:34:130
30369990,2cyclictest624-21ssh21:22:030
30369990,2cyclictest3521-21ssh00:11:370
30369990,2cyclictest31255-21ssh00:06:310
30369990,2cyclictest25943-21wc20:05:140
30369990,2cyclictest22078-21ssh00:33:210
30369990,0cyclictest0-21swapper/021:02:270
30489988,0cyclictest66850irq/125-lan20:45:023
30489988,0cyclictest66850irq/125-lan20:35:183
30489986,2cyclictest771-21snmpd19:29:503
30449988,0cyclictest771-21snmpd23:32:122
30449988,0cyclictest771-21snmpd22:57:382
30449988,0cyclictest771-21snmpd21:20:282
30449988,0cyclictest771-21snmpd00:31:022
30449988,0cyclictest769-21lldpd22:20:262
30449988,0cyclictest769-21lldpd20:54:182
30449988,0cyclictest769-21lldpd19:30:112
30449987,1cyclictest771-21snmpd23:22:212
30449987,1cyclictest771-21snmpd21:40:262
30449987,1cyclictest6985-21cut19:20:002
30449987,1cyclictest524-21avahi-daemon22:53:002
30449987,1cyclictest255-20systemd-journal00:23:362
30449986,2cyclictest771-21snmpd23:25:442
30449986,2cyclictest771-21snmpd22:29:402
30399988,0cyclictest771-21snmpd19:22:311
30399988,0cyclictest769-21lldpd20:49:021
30399988,0cyclictest769-21lldpd20:43:221
30399988,0cyclictest769-21lldpd00:09:411
30399987,1cyclictest771-21snmpd20:17:511
30399987,1cyclictest524-21avahi-daemon00:32:021
30399987,1cyclictest517-21systemd-resolve22:00:001
30399986,2cyclictest771-21snmpd23:52:091
30399986,2cyclictest771-21snmpd21:08:281
30399986,2cyclictest771-21snmpd00:27:391
30369988,0cyclictest771-21snmpd23:00:050
30369988,0cyclictest771-21snmpd22:09:100
30369988,0cyclictest771-21snmpd19:24:340
30369988,0cyclictest769-21lldpd22:18:180
30369988,0cyclictest769-21lldpd21:53:240
30369987,1cyclictest771-21snmpd23:15:380
30369987,1cyclictest769-21lldpd23:34:070
30369987,1cyclictest517-21systemd-resolve23:21:470
30369986,2cyclictest769-21lldpd23:14:180
30369986,2cyclictest749-21ntpd20:18:210
30369986,2cyclictest749-21ntpd19:11:290
30369985,3cyclictest524-21avahi-daemon23:40:310
30369985,3cyclictest31787-21cpuspeed_turbos21:55:120
30369984,4cyclictest771-21snmpd23:36:060
30369984,4cyclictest771-21snmpd19:36:160
30369984,3cyclictest716-21apt-get22:30:110
30369983,5cyclictest550-21NetworkManager23:51:060
30369982,3cyclictest771-21snmpd19:16:300
30369982,3cyclictest15276-21sed20:55:210
30369982,2cyclictest21456-21ssh23:56:500
30369981,3cyclictest27672-21ssh21:16:130
30369980,8cyclictest24122-21load22:20:170
30369980,4cyclictest20253-21apt22:50:090
30369980,4cyclictest11518-21ntp_statesmunin00:20:170
30369980,3cyclictest31934-21interrupts20:20:140
30369980,3cyclictest28215-21/usr/sbin/munin20:10:210
30449977,0cyclictest771-21snmpd23:55:152
30449977,0cyclictest771-21snmpd19:10:272
30449977,0cyclictest769-21lldpd22:46:242
30449977,0cyclictest769-21lldpd22:17:042
30449977,0cyclictest769-21lldpd20:37:222
30449977,0cyclictest769-21lldpd19:46:192
30449976,1cyclictest524-21avahi-daemon22:31:432
30449976,1cyclictest12809-21fschecks_time21:35:182
30449975,2cyclictest771-21snmpd22:07:362
30449975,2cyclictest771-21snmpd22:02:242
30449975,2cyclictest771-21snmpd21:12:282
30449975,2cyclictest771-21snmpd20:42:212
30449975,2cyclictest771-21snmpd00:02:282
30449975,2cyclictest6024-21cat22:35:152
30449972,4cyclictest9188-21cat19:25:002
30399977,0cyclictest771-21snmpd23:11:021
30399977,0cyclictest771-21snmpd20:25:051
30399977,0cyclictest769-21lldpd23:37:551
30399977,0cyclictest769-21lldpd22:53:031
30399977,0cyclictest769-21lldpd22:25:561
30399977,0cyclictest769-21lldpd22:22:391
30399977,0cyclictest769-21lldpd20:55:161
30399976,1cyclictest771-21snmpd21:55:511
30399976,1cyclictest771-21snmpd21:37:331
30399976,1cyclictest771-21snmpd19:42:131
30399976,1cyclictest771-21snmpd00:11:411
30399976,1cyclictest21646-21fschecks_time19:55:131
30399975,2cyclictest771-21snmpd23:55:401
30399975,2cyclictest771-21snmpd00:36:201
30399974,3cyclictest771-21snmpd22:11:151
30399973,3cyclictest19657-21sed22:15:171
30399972,5cyclictest550-21NetworkManager20:08:411
30369977,0cyclictest769-21lldpd19:33:110
30369972,5cyclictest1295-21cut20:25:130
30369972,2cyclictest771-21snmpd19:54:530
30369972,2cyclictest24167-21ssh00:35:140
30369972,1cyclictest0-21swapper/021:35:180
30369971,3cyclictest18421-21systemctl21:40:200
30369971,2cyclictest12790-21ntp_kernel_pll_20:50:170
30369970,7cyclictest0-21swapper/022:26:280
30369970,3cyclictest5949-21ssh00:15:120
30369970,3cyclictest5023-21ntpq22:00:170
30369970,3cyclictest4525-21ssh23:06:350
30369970,3cyclictest4004-21/usr/sbin/munin20:30:180
30369970,3cyclictest26788-21ssh22:55:400
30369970,3cyclictest24897-21ntpq00:00:210
30369970,3cyclictest23458-21timerwakeupswit21:45:210
30369970,3cyclictest20415-21df21:10:130
30369970,3cyclictest15303-21ssh22:45:090
30369970,3cyclictest15038-21diskstats00:25:140
30369970,3cyclictest11538-21ssh23:47:220
30369970,3cyclictest10629-21diskstats22:40:140
30369970,3cyclictest10182-21df20:45:130
30449966,0cyclictest771-21snmpd23:36:222
30449966,0cyclictest769-21lldpd20:28:362
30449966,0cyclictest769-21lldpd19:54:032
30449965,1cyclictest771-21snmpd20:21:352
30449965,1cyclictest29370-21gltestperf00:05:142
30449964,2cyclictest771-21snmpd00:18:462
30449964,2cyclictest550-21NetworkManager00:39:092
30449963,3cyclictest771-21snmpd21:16:042
30449963,3cyclictest0-21swapper/223:02:432
30449963,2cyclictest255-20systemd-journal00:26:532
30449963,2cyclictest25349-21ssh21:48:062
30449963,2cyclictest2029-21ssh21:57:192
30449962,3cyclictest6593-21ssh23:09:502
30449962,3cyclictest5817-21ssh21:27:302
30449962,3cyclictest5023-21ssh23:40:212
30449962,3cyclictest255-20systemd-journal19:37:322
30449962,3cyclictest1938-21sed00:10:162
30449962,3cyclictest11880-21ssh23:15:102
30399966,0cyclictest771-21snmpd21:22:351
30399966,0cyclictest771-21snmpd20:39:421
30399966,0cyclictest771-21snmpd19:45:251
30399966,0cyclictest769-21lldpd23:20:301
30399966,0cyclictest769-21lldpd20:04:401
30399966,0cyclictest769-21lldpd19:30:551
30399966,0cyclictest769-21lldpd19:12:171
30399966,0cyclictest524-21avahi-daemon22:36:161
30399965,1cyclictest9265-21cat23:45:141
30399965,1cyclictest771-21snmpd22:44:101
30399965,1cyclictest771-21snmpd20:52:451
30399965,1cyclictest524-21avahi-daemon20:33:381
30399965,1cyclictest255-20systemd-journal21:33:041
30399965,1cyclictest19444-21cpuspeed_turbos19:50:121
30399964,2cyclictest9588-21cat22:05:171
30399964,2cyclictest771-21snmpd23:44:251
30399964,2cyclictest771-21snmpd23:34:031
30399964,2cyclictest771-21snmpd21:48:261
30399964,2cyclictest771-21snmpd19:36:551
30399964,2cyclictest11109-21cat00:20:141
30399963,3cyclictest550-21NetworkManager00:18:321
30399962,3cyclictest27133-21sh21:15:211
30399962,3cyclictest21357-21ssh21:10:201
30399960,2cyclictest2911650ssh21:51:311
30399960,2cyclictest2503350ssh22:55:051
30369964,2cyclictest749-21ntpd19:40:070
30369960,2cyclictest17517-21df_inode19:45:130
30449954,1cyclictest769-21lldpd20:07:402
30449954,1cyclictest21678-21gltestperf19:55:132
30449954,1cyclictest0-21swapper/220:05:002
30449953,2cyclictest771-21snmpd20:32:302
30449952,2cyclictest9343-21sh23:11:412
30449952,2cyclictest8814-21apt-get23:45:112
30449952,2cyclictest18545-21interrupts21:05:132
30449952,2cyclictest13911-21apt-get22:10:142
30449952,2cyclictest1131-21kworker/u16:0+nfsiod23:53:212
30449952,2cyclictest10984-21ps20:45:202
30449951,3cyclictest9764-21ssh21:31:122
30449951,3cyclictest10743-21seq19:26:342
30399955,0cyclictest524-21avahi-daemon20:23:231
30399954,1cyclictest0-21swapper/121:40:221
30399953,2cyclictest550-21NetworkManager23:25:281
30399952,3cyclictest517-21systemd-resolve19:26:321
30399952,3cyclictest5170-21ssh21:26:321
30399952,3cyclictest29322-21sed20:15:001
30399952,2cyclictest24481-21hddtemp_smartct00:00:181
30399952,2cyclictest17306-21ssh22:46:061
30399952,2cyclictest1554-21awk22:30:161
30399952,2cyclictest11868-21ssh23:15:091
30399951,3cyclictest32166-21ssh23:01:331
30399951,3cyclictest2610-21fw_forwarded_lo23:05:161
30449942,2cyclictest14299-21tune2fs20:55:132
30449941,2cyclictest255-20systemd-journal21:00:112
30449940,4cyclictest16124-21ntpq19:40:162
30449940,3cyclictest29612-21diskstats20:15:142
30449940,3cyclictest27994-21memory20:10:192
30399940,4cyclictest5993-21ntpq19:15:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional