You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-19 - 10:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Jul 19, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12070991919,0cyclictest66850irq/125-lan22:40:150
12070991918,1cyclictest66850irq/125-lan23:40:160
12070991918,1cyclictest66850irq/125-lan22:58:250
12070991918,1cyclictest66850irq/125-lan22:30:200
12070991818,0cyclictest66850irq/125-lan23:26:060
12070991818,0cyclictest66850irq/125-lan22:28:350
12070991818,0cyclictest66850irq/125-lan21:59:280
12070991818,0cyclictest66850irq/125-lan21:48:520
12070991818,0cyclictest66850irq/125-lan21:23:260
12070991818,0cyclictest66850irq/125-lan21:16:210
12070991818,0cyclictest66850irq/125-lan00:10:230
12070991818,0cyclictest66850irq/125-lan00:06:560
12070991817,1cyclictest66850irq/125-lan23:22:140
12070991817,1cyclictest66850irq/125-lan23:15:460
12070991817,1cyclictest66850irq/125-lan23:06:070
12070991817,1cyclictest66850irq/125-lan22:35:520
12070991817,1cyclictest66850irq/125-lan22:20:190
12070991817,1cyclictest66850irq/125-lan22:10:160
12070991817,1cyclictest66850irq/125-lan21:50:210
12070991817,1cyclictest66850irq/125-lan21:37:390
12070991817,1cyclictest66850irq/125-lan21:31:090
12070991817,1cyclictest66850irq/125-lan20:41:110
12086991717,0cyclictest771-21snmpd21:51:243
12070991717,0cyclictest66850irq/125-lan23:45:160
12070991717,0cyclictest66850irq/125-lan22:50:110
12070991717,0cyclictest66850irq/125-lan22:15:420
12070991717,0cyclictest66850irq/125-lan22:00:530
12070991717,0cyclictest66850irq/125-lan21:43:210
12070991717,0cyclictest66850irq/125-lan21:11:390
12070991717,0cyclictest66850irq/125-lan00:25:040
12070991717,0cyclictest66850irq/125-lan00:15:430
12070991716,1cyclictest66850irq/125-lan23:57:340
12070991716,1cyclictest66850irq/125-lan23:51:000
12070991716,1cyclictest66850irq/125-lan23:35:050
12070991716,1cyclictest66850irq/125-lan23:30:210
12070991716,1cyclictest66850irq/125-lan23:10:100
12070991716,1cyclictest66850irq/125-lan23:00:010
12070991716,1cyclictest66850irq/125-lan22:07:580
12070991716,1cyclictest66850irq/125-lan21:25:150
12070991716,1cyclictest66850irq/125-lan20:35:210
12070991716,1cyclictest66850irq/125-lan20:15:190
12070991716,1cyclictest66850irq/125-lan19:20:170
12070991716,1cyclictest66850irq/125-lan19:10:130
12070991716,1cyclictest66850irq/125-lan00:37:110
12086991616,0cyclictest771-21snmpd21:26:513
12086991616,0cyclictest771-21snmpd21:08:203
12086991616,0cyclictest771-21snmpd20:48:423
12086991614,2cyclictest749-21ntpd23:01:443
12070991616,0cyclictest66850irq/125-lan22:45:320
12070991616,0cyclictest66850irq/125-lan20:50:470
12070991616,0cyclictest66850irq/125-lan19:33:060
12070991616,0cyclictest66850irq/125-lan00:20:230
12070991615,1cyclictest66850irq/125-lan20:45:220
12070991615,1cyclictest66850irq/125-lan19:50:180
12070991615,1cyclictest66850irq/125-lan19:45:200
12070991615,1cyclictest66850irq/125-lan00:30:210
12070991615,1cyclictest66850irq/125-lan00:00:180
12086991515,0cyclictest771-21snmpd22:22:383
12086991515,0cyclictest771-21snmpd20:31:393
12086991515,0cyclictest771-21snmpd20:22:313
12086991515,0cyclictest771-21snmpd19:58:033
12086991515,0cyclictest771-21snmpd19:43:503
12086991515,0cyclictest771-21snmpd00:03:533
12086991514,1cyclictest749-21ntpd22:34:353
12086991513,2cyclictest771-21snmpd19:49:243
12080991515,0cyclictest771-21snmpd22:40:502
12070991514,1cyclictest66850irq/125-lan20:30:160
12086991414,0cyclictest771-21snmpd23:57:293
12086991414,0cyclictest771-21snmpd23:53:233
12086991414,0cyclictest771-21snmpd23:06:003
12086991414,0cyclictest771-21snmpd22:57:153
12086991414,0cyclictest771-21snmpd22:52:123
12086991414,0cyclictest771-21snmpd22:14:403
12086991414,0cyclictest771-21snmpd21:48:543
12086991414,0cyclictest771-21snmpd21:38:573
12086991414,0cyclictest771-21snmpd21:02:233
12086991414,0cyclictest771-21snmpd20:44:403
12086991414,0cyclictest771-21snmpd20:35:133
12086991414,0cyclictest771-21snmpd20:18:023
12086991414,0cyclictest771-21snmpd20:12:003
12086991414,0cyclictest771-21snmpd19:28:113
12086991414,0cyclictest771-21snmpd19:20:473
12086991414,0cyclictest771-21snmpd19:17:593
12086991414,0cyclictest771-21snmpd00:29:003
12086991414,0cyclictest771-21snmpd00:11:173
12070991414,0cyclictest66850irq/125-lan21:00:170
12070991413,1cyclictest66850irq/125-lan20:56:160
12070991413,1cyclictest66850irq/125-lan20:10:150
12070991413,1cyclictest66850irq/125-lan19:35:200
12086991313,0cyclictest771-21snmpd23:30:133
12086991313,0cyclictest771-21snmpd22:03:463
12086991313,0cyclictest771-21snmpd19:54:573
12070991313,0cyclictest66850irq/125-lan19:25:220
12070991312,1cyclictest66850irq/125-lan21:05:160
12070991312,1cyclictest66850irq/125-lan19:56:320
12070991312,1cyclictest66850irq/125-lan19:40:190
12070991212,0cyclictest66850irq/125-lan20:25:480
12070991212,0cyclictest66850irq/125-lan20:20:200
12070991211,1cyclictest66850irq/125-lan20:05:150
12070991211,1cyclictest66850irq/125-lan20:00:140
1208699119,2cyclictest749-21ntpd21:56:273
12086991110,1cyclictest749-21ntpd23:15:443
12080991010,0cyclictest771-21snmpd23:02:302
12080991010,0cyclictest771-21snmpd22:36:382
1207099109,1cyclictest66850irq/125-lan19:15:210
120869998,1cyclictest769-21lldpd00:08:163
120869998,1cyclictest524-21avahi-daemon00:20:183
120809999,0cyclictest771-21snmpd23:55:142
120809999,0cyclictest771-21snmpd22:24:442
120809998,1cyclictest771-21snmpd19:57:152
120809998,1cyclictest771-21snmpd00:29:452
120809998,1cyclictest524-21avahi-daemon22:07:322
120749998,1cyclictest771-21snmpd22:59:301
120749998,1cyclictest771-21snmpd19:44:141
120749998,1cyclictest524-21avahi-daemon19:15:121
120869988,0cyclictest771-21snmpd20:53:353
120869988,0cyclictest771-21snmpd20:02:423
120869988,0cyclictest769-21lldpd23:11:223
120869988,0cyclictest769-21lldpd19:33:013
120869987,1cyclictest771-21snmpd22:35:173
120869987,1cyclictest524-21avahi-daemon21:11:413
120869986,2cyclictest771-21snmpd19:12:023
120869986,2cyclictest749-21ntpd23:37:263
120869986,2cyclictest749-21ntpd22:47:093
120869986,2cyclictest749-21ntpd21:33:003
120869986,2cyclictest749-21ntpd19:35:203
120869982,6cyclictest550-21NetworkManager21:43:213
120809988,0cyclictest9411-21cat20:20:142
120809988,0cyclictest771-21snmpd23:10:452
120809988,0cyclictest771-21snmpd23:06:032
120809988,0cyclictest771-21snmpd22:29:592
120809988,0cyclictest771-21snmpd20:34:402
120809988,0cyclictest771-21snmpd20:28:432
120809988,0cyclictest771-21snmpd00:38:542
120809988,0cyclictest769-21lldpd22:54:472
120809988,0cyclictest769-21lldpd19:36:452
120809987,1cyclictest771-21snmpd21:20:472
120809987,1cyclictest771-21snmpd20:47:582
120809987,1cyclictest255-20systemd-journal00:02:352
120809986,2cyclictest771-21snmpd22:55:422
120749988,0cyclictest771-21snmpd22:51:391
120749988,0cyclictest771-21snmpd22:08:311
120749988,0cyclictest771-21snmpd21:55:011
120749988,0cyclictest771-21snmpd21:31:181
120749988,0cyclictest771-21snmpd00:39:161
120749988,0cyclictest769-21lldpd23:50:391
120749988,0cyclictest769-21lldpd22:26:111
120749988,0cyclictest769-21lldpd22:00:171
120749988,0cyclictest769-21lldpd21:02:141
120749988,0cyclictest769-21lldpd19:56:571
120749988,0cyclictest769-21lldpd19:29:521
120749987,1cyclictest769-21lldpd19:21:281
120749987,1cyclictest524-21avahi-daemon23:29:391
120749987,1cyclictest524-21avahi-daemon22:30:161
120749986,2cyclictest8862-21cat23:15:161
120749986,2cyclictest771-21snmpd20:08:451
120749986,2cyclictest771-21snmpd19:14:561
120869977,0cyclictest769-21lldpd00:31:453
120869976,1cyclictest524-21avahi-daemon21:19:013
120869975,2cyclictest749-21ntpd23:25:123
120869975,2cyclictest749-21ntpd22:16:233
120869975,1cyclictest31936-21kworker/u16:3+events_unbound21:20:143
120869973,3cyclictest16404-21diskstats22:25:103
120809977,0cyclictest771-21snmpd23:46:142
120809977,0cyclictest771-21snmpd22:49:512
120809977,0cyclictest771-21snmpd22:11:312
120809977,0cyclictest771-21snmpd21:41:572
120809977,0cyclictest771-21snmpd21:10:562
120809977,0cyclictest771-21snmpd21:09:382
120809977,0cyclictest771-21snmpd21:02:022
120809977,0cyclictest771-21snmpd20:35:042
120809977,0cyclictest771-21snmpd20:05:302
120809977,0cyclictest771-21snmpd19:32:252
120809977,0cyclictest771-21snmpd00:21:122
120809977,0cyclictest771-21snmpd00:09:532
120809977,0cyclictest769-21lldpd20:10:092
120809977,0cyclictest4202-21cat00:10:162
120809977,0cyclictest31119-21cat23:35:152
120809976,1cyclictest771-21snmpd19:26:112
120809976,1cyclictest749-21ntpd23:31:342
120809976,1cyclictest749-21ntpd22:33:152
120809976,1cyclictest749-21ntpd21:30:342
120809975,2cyclictest771-21snmpd23:27:132
120809975,2cyclictest749-21ntpd23:21:522
120809975,2cyclictest4439-21grep21:45:162
120749977,0cyclictest771-21snmpd23:07:001
120749977,0cyclictest771-21snmpd22:48:051
120749977,0cyclictest771-21snmpd22:44:591
120749977,0cyclictest771-21snmpd21:15:141
120749977,0cyclictest771-21snmpd19:32:531
120749977,0cyclictest771-21snmpd00:32:421
120749977,0cyclictest769-21lldpd22:19:431
120749977,0cyclictest769-21lldpd21:25:401
120749977,0cyclictest769-21lldpd20:51:341
120749977,0cyclictest769-21lldpd20:38:421
120749977,0cyclictest517-21systemd-resolve20:10:361
120749977,0cyclictest517-21systemd-resolve00:12:581
120749976,1cyclictest771-21snmpd21:06:321
120749976,1cyclictest524-21avahi-daemon23:10:351
120749976,1cyclictest524-21avahi-daemon23:02:311
120749976,1cyclictest524-21avahi-daemon21:52:061
120749976,1cyclictest517-21systemd-resolve22:23:221
120749976,1cyclictest517-21systemd-resolve21:22:211
120749976,1cyclictest255-20systemd-journal00:02:211
120749975,2cyclictest771-21snmpd23:42:071
120749975,2cyclictest771-21snmpd23:21:501
120749975,2cyclictest771-21snmpd21:41:541
120749975,2cyclictest771-21snmpd20:56:051
120749975,2cyclictest771-21snmpd00:26:181
120749975,2cyclictest771-21snmpd00:15:331
120749975,2cyclictest28858-21cat19:50:151
120869965,1cyclictest749-21ntpd23:45:203
120869965,1cyclictest749-21ntpd23:24:583
120869965,0cyclictest771-21snmpd00:17:183
120869964,2cyclictest749-21ntpd00:36:513
120869963,2cyclictest586-21rs:main3
120869962,3cyclictest255-20systemd-journal22:05:423
120809966,0cyclictest771-21snmpd23:16:542
120809966,0cyclictest771-21snmpd20:54:502
120809966,0cyclictest771-21snmpd20:15:182
120809966,0cyclictest771-21snmpd20:00:142
120809966,0cyclictest769-21lldpd22:04:572
120809966,0cyclictest769-21lldpd00:15:132
120809965,1cyclictest749-21ntpd21:15:102
120809965,1cyclictest749-21ntpd00:32:032
120809965,1cyclictest524-21avahi-daemon21:36:352
120809964,2cyclictest771-21snmpd21:50:032
120809964,2cyclictest749-21ntpd23:50:572
120809964,2cyclictest749-21ntpd21:55:532
120809964,2cyclictest749-21ntpd21:25:442
120809963,3cyclictest16616-21perl19:20:182
120809963,2cyclictest255-20systemd-journal23:42:452
120809962,3cyclictest6420-21awk22:15:212
120749966,0cyclictest769-21lldpd21:45:141
120749966,0cyclictest524-21avahi-daemon21:14:411
120749966,0cyclictest524-21avahi-daemon20:46:531
120749966,0cyclictest524-21avahi-daemon19:38:121
120749962,3cyclictest255-20systemd-journal20:16:381
120749962,3cyclictest255-20systemd-journal19:45:531
120749962,3cyclictest1461-21ssh00:07:101
120869952,2cyclictest1360-21entropy22:40:143
120869950,3cyclictest4242-21latency_hist20:10:003
120809955,0cyclictest769-21lldpd19:10:262
120809954,1cyclictest771-21snmpd19:47:032
120809952,3cyclictest550-21NetworkManager19:51:262
120809952,2cyclictest15574-21unixbench_multi19:15:212
120809950,4cyclictest24359-21ntp_states20:55:192
120749955,0cyclictest771-21snmpd20:26:071
120749955,0cyclictest769-21lldpd21:35:081
120749954,1cyclictest771-21snmpd20:43:431
120749953,2cyclictest9970-21rt-features20:20:181
120749952,2cyclictest30748-21ssh22:37:021
120749952,2cyclictest24978-21ssh23:30:061
120749952,2cyclictest12696-21ssh23:47:381
120749950,4cyclictest31599-21ntpq23:35:191
120869940,4cyclictest24239-21ntpq20:55:183
120869940,3cyclictest11388-21hddtemp_smartct20:25:163
120809941,2cyclictest24660-21fschecks_time19:40:152
120809940,3cyclictest17107-21grep20:40:132
120749943,1cyclictest21355-21ntpq23:55:181
120749942,2cyclictest331ktimers/120:00:131
120749942,2cyclictest31882-21apt-get22:10:161
120749941,2cyclictest14106-21sed20:30:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional