You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 23:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Mar 29, 2026 12:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2180801997058,7cyclictest0-21swapper/112:30:271
2180802995330,18cyclictest421-21dbus-daemon11:05:372
218080199522,38cyclictest2357578-21ssh12:00:271
2180802995140,7cyclictest1-21systemd10:50:012
2180801995041,6cyclictest2340127-21awk11:40:011
2180802994940,6cyclictest2270642-21latency_hist10:15:022
2180801994939,6cyclictest2274644-21munin-plugin-st10:20:011
2180802994838,6cyclictest2358150-21unixbench_multi12:00:382
2180802994838,5cyclictest0-21swapper/212:25:302
2180800994811,29cyclictest2308494-21ssh11:00:270
2180802994734,7cyclictest420-21avahi-daemon09:14:052
2180801994738,4cyclictest0-21swapper/109:00:011
2180801994738,4cyclictest0-21swapper/109:00:011
2180801994636,7cyclictest2242084-21aten_r2power_po09:40:181
2180801994636,5cyclictest0-21swapper/109:45:281
218080199462,39cyclictest2369745-21ssh12:15:251
2180800994639,5cyclictest421-21dbus-daemon08:30:010
2180802994536,5cyclictest2311533-21ssh11:05:012
2180800994539,4cyclictest2340849-21uniq11:40:220
2180800994537,5cyclictest1319609-21cups-browsed11:31:120
218080299442,17cyclictest2333190-21ssh11:30:342
2180801994434,6cyclictest421-21dbus-daemon10:40:011
2180802994334,3cyclictest0-21swapper/211:15:322
2180801994334,6cyclictest2356444-21ssh12:00:011
2180800994336,4cyclictest2345737-21sshd11:45:410
2180800994334,7cyclictest12350irq/32-eth012:10:270
2180800994330,10cyclictest2286948-21debian-sa110:35:010
2180800994330,10cyclictest2286948-21debian-sa110:35:010
2180802994231,7cyclictest421-21dbus-daemon10:24:092
2180800994228,10cyclictest12350irq/32-eth012:05:340
2180802994133,5cyclictest2283525-21fschecks_count10:30:202
2180802994133,5cyclictest2283525-21fschecks_count10:30:202
2180801994134,4cyclictest2200799-21fschecks_time08:15:201
2180801994133,5cyclictest2295146-21ssh10:45:011
2180801994133,5cyclictest2283944-21ntp_kernel_pll_10:30:271
2180801994133,5cyclictest2283944-21ntp_kernel_pll_10:30:271
2180800994135,4cyclictest2180799-21cyclictest09:50:490
2180800994135,4cyclictest2180799-21cyclictest09:50:490
2180800994128,8cyclictest2386187-21sh12:35:270
2180802994033,4cyclictest2348296-21systemd11:50:022
2180801994032,4cyclictest2229158-21cron09:25:011
2180801994029,7cyclictest420-21avahi-daemon08:45:331
2180801994028,8cyclictest538-21wpa_supplicant09:33:221
2180800994032,5cyclictest2275535-21latency10:20:220
218080099401,35cyclictest2216778-21diskmemload11:15:260
2180802993934,2cyclictest0-21swapper/210:40:002
218080299393,25cyclictest2300312-21ssh10:50:272
2180802993931,5cyclictest2186789-21cron07:30:012
2180802993927,8cyclictest548-21gdbus07:50:012
2180802993927,8cyclictest548-21gdbus07:50:012
218080299392,26cyclictest2255240-21ssh09:55:322
218080199393,29cyclictest2238576-21ssh09:35:311
2180801993931,5cyclictest2377765-21wc12:25:241
2180801993930,6cyclictest1055-21mta-sts-daemon08:05:291
2180800993933,4cyclictest2212295-21systemd-run08:55:010
218080299383,31cyclictest2337092-21ssh11:35:262
2180802993829,6cyclictest2368812-21latency_hist12:15:022
2180802993826,9cyclictest2324734-21missed_timers11:20:242
2180802993826,9cyclictest2324734-21missed_timers11:20:242
218080199383,25cyclictest2275728-21ssh10:20:251
2180800993832,3cyclictest2206296-21kworker/u8:14-rpciod11:55:300
2180800993831,4cyclictest12350irq/32-eth009:45:320
2180800993830,6cyclictest420-21avahi-daemon10:00:280
218080099382,30cyclictest2267774-21ssh10:10:350
2180800993817,15cyclictest12350irq/32-eth011:50:290
218080099381,32cyclictest1319609-21cups-browsed12:25:300
2180802993729,5cyclictest2229162-21latency_hist09:25:012
2180802993729,5cyclictest0-21swapper/208:00:012
2180802993728,6cyclictest420-21avahi-daemon09:55:012
2180802993728,6cyclictest420-21avahi-daemon09:55:012
2180802993726,8cyclictest2207773-21/usr/sbin/munin08:40:022
2180801993730,3cyclictest0-21swapper/109:10:271
2180801993726,8cyclictest1-21systemd09:10:011
2180800993729,5cyclictest1319609-21cups-browsed11:38:540
2180800993729,5cyclictest1319609-21cups-browsed10:15:520
2180800993717,8cyclictest2301929-21sh10:52:260
2180802993629,4cyclictest1054-21mta-sts-daemon11:10:202
2180802993626,7cyclictest0-21swapper/209:40:012
2180802993626,6cyclictest1319609-21cups-browsed12:17:152
218080299362,5cyclictest171rcu_preempt07:40:242
2180802993625,5cyclictest0-21swapper/211:43:512
218080299362,27cyclictest2279993-21ssh10:25:332
218080299361,25cyclictest2222167-21ssh09:15:332
2180801993628,5cyclictest420-21avahi-daemon10:45:291
2180801993628,5cyclictest2215437-21latency_hist09:05:021
2180801993628,4cyclictest0-21swapper/110:25:271
2180801993628,3cyclictest0-21swapper/111:30:301
2180801993627,6cyclictest2303290-21cron10:55:011
2180801993627,4cyclictest0-21swapper/110:05:361
2180800993624,7cyclictest12350irq/32-eth009:55:310
2180802993529,3cyclictest2189793-21plymouth07:40:012
2180802993528,4cyclictest0-21swapper/212:40:012
2180802993524,7cyclictest1319606-21cupsd11:29:222
2180801993527,5cyclictest2389167-21cron12:40:011
2180801993524,8cyclictest2329584-21unixbench_multi11:25:411
218080199352,26cyclictest2304463-21ssh10:55:311
218080099358,23cyclictest2312595-21ssh11:05:280
2180800993530,2cyclictest0-21swapper/010:45:160
2180800993528,4cyclictest2328046-21latency_hist11:25:030
2180800993528,4cyclictest2328046-21latency_hist11:25:030
2180800993526,7cyclictest2381893-21latency12:30:230
2180800993522,11cyclictest2196177-21cut08:00:190
2180800993520,10cyclictest12350irq/32-eth012:15:410
2180800993517,13cyclictest12350irq/32-eth010:40:340
2180802993423,7cyclictest420-21avahi-daemon09:43:032
2180802993422,6cyclictest0-21swapper/207:15:262
218080299342,24cyclictest2361728-21ssh12:05:312
2180801993429,3cyclictest0-21swapper/109:20:011
2180801993427,4cyclictest0-21swapper/111:15:011
2180801993427,3cyclictest0-21swapper/111:50:311
2180801993427,3cyclictest0-21swapper/111:15:151
2180801993426,5cyclictest2311506-21latency_hist11:05:011
2180801993426,5cyclictest2268170-21unixbench_multi10:10:401
2180801993425,4cyclictest0-21swapper/108:10:261
2180801993422,8cyclictest2368696-21grep12:15:011
218080099344,24cyclictest2222105-21ssh09:15:300
2180800993429,3cyclictest1055-21mta-sts-daemon09:25:270
2180800993429,3cyclictest0-21swapper/008:20:010
2180800993428,4cyclictest1054-21mta-sts-daemon10:35:260
2180800993426,5cyclictest420-21avahi-daemon09:32:030
2180800993426,5cyclictest2210690-21head08:45:370
2180800993423,8cyclictest1-21systemd07:25:010
2180802993325,5cyclictest2197285-21cron08:05:002
2180802993325,5cyclictest1319609-21cups-browsed11:50:292
2180802993325,3cyclictest0-21swapper/212:20:192
2180802993324,4cyclictest0-21swapper/207:10:252
2180802993323,6cyclictest0-21swapper/210:05:262
2180802993322,7cyclictest1319609-21cups-browsed10:03:132
218080299332,23cyclictest1319606-21cupsd07:50:252
218080299331,23cyclictest2382210-21ssh12:30:322
2180801993328,3cyclictest2299494-21kworker/u8:10-xprtiod11:20:161
2180801993328,3cyclictest2299494-21kworker/u8:10-xprtiod11:20:161
2180801993327,2cyclictest0-21swapper/112:20:331
2180801993325,5cyclictest2191184-21unixbench_multi07:40:371
2180801993325,4cyclictest217429650kworker/u8:10+xprtiod08:05:011
2180801993314,14cyclictest420-21avahi-daemon11:40:331
2180800993319,11cyclictest2218312-21awk09:10:390
2180802993226,3cyclictest0-21swapper/208:49:432
2180802993225,3cyclictest0-21swapper/211:55:302
218080299322,22cyclictest2304513-21ssh10:55:322
2180802993219,4cyclictest2225068-21kworker/u8:5-xprtiod09:30:262
2180801993224,5cyclictest2262320-21grep10:05:011
2180801993222,8cyclictest2345819-21timerwakeupswit11:45:411
2180800993225,4cyclictest2279652-21latency10:25:240
2180800993217,10cyclictest2263307-21/usr/sbin/munin10:05:240
2180802993124,3cyclictest0-21swapper/210:15:172
2180802993124,3cyclictest0-21swapper/209:30:022
2180802993122,7cyclictest2205922-21switchtime08:30:382
2180802993122,7cyclictest2204588-21timerwakeupswit08:25:382
2180802993122,6cyclictest420-21avahi-daemon08:55:222
2180802993122,6cyclictest420-21avahi-daemon08:55:222
218080199313,20cyclictest2255038-21ssh09:55:241
2180801993124,4cyclictest0-21swapper/109:30:011
2180801993123,5cyclictest2206270-21cron08:35:011
2180800993123,4cyclictest0-21swapper/009:23:260
218080099312,25cyclictest2316868-21ssh11:10:340
2180800993121,8cyclictest2183787-21munin-run07:20:010
2180800993117,12cyclictest2208320-21fschecks_time08:40:200
2180802993022,5cyclictest2198807-21systemd-run08:10:022
2180802993022,3cyclictest0-21swapper/209:45:272
218080299302,21cyclictest2292330-21ssh10:40:352
218080099304,21cyclictest440-21thd12:20:290
2180800993024,3cyclictest0-21swapper/010:55:180
2180800993017,10cyclictest624-21lldpd09:05:280
2180801992922,3cyclictest0-21swapper/109:51:411
2180801992922,3cyclictest0-21swapper/109:51:411
218080199292,21cyclictest2312753-21ssh11:05:331
2180801992921,5cyclictest2186703-21unixbench_singl07:25:371
2180801992919,7cyclictest2194171-21unixbench_multi07:50:371
2180800992923,3cyclictest0-21swapper/008:00:020
218080099292,23cyclictest2242808-21ssh09:40:320
2180800992921,6cyclictest2241617-21latency_hist09:40:020
218080099291,23cyclictest2328962-21ssh11:25:290
2180800992911,13cyclictest2357370-21memory12:00:220
218080299284,2cyclictest0-21swapper/208:21:412
218080199283,19cyclictest428-21mta-sts-daemon08:40:341
2180801992822,3cyclictest0-21swapper/108:55:011
2180801992820,5cyclictest261-21systemd-journal07:50:021
2180801992820,5cyclictest261-21systemd-journal07:50:021
2180800992821,5cyclictest420-21avahi-daemon07:35:210
2180800992820,6cyclictest2182284-21munin-run07:15:010
2180800992816,9cyclictest0-21swapper/007:40:190
218080199272,19cyclictest1319609-21cups-browsed08:20:321
2180801992720,4cyclictest2207763-21cron08:40:011
2180801992719,5cyclictest624-21lldpd12:05:101
2180801992717,7cyclictest2195652-21unixbench_multi07:55:381
2180800992723,2cyclictest0-21swapper/009:00:000
2180800992723,2cyclictest0-21swapper/009:00:000
2180800992722,3cyclictest151ktimers/008:10:330
2180800992718,7cyclictest1054-21mta-sts-daemon07:50:150
2180802992620,2cyclictest0-21swapper/209:10:002
2180802992618,5cyclictest2208056-21aten_r2power_cu08:40:172
2180802992617,5cyclictest0-21swapper/207:30:312
2180801992619,5cyclictest1051-21mta-sts-daemon07:15:381
2180801992619,4cyclictest0-21swapper/107:40:011
2180800992620,4cyclictest12450irq/33-eth008:35:310
2180800992619,5cyclictest2186813-21systemd-run07:30:010
218080399251,6cyclictest0-21swapper/308:30:303
218080399251,5cyclictest0-21swapper/308:15:323
218080399251,4cyclictest0-21swapper/312:00:273
2180802992518,4cyclictest0-21swapper/208:20:012
2180801992517,5cyclictest2187522-21missed_timers07:30:241
2180800992516,7cyclictest2188246-21sshd07:30:400
2180800992513,2cyclictest0-21swapper/007:45:310
2180800992513,2cyclictest0-21swapper/007:45:310
218080399242,3cyclictest0-21swapper/307:55:273
218080399241,5cyclictest0-21swapper/311:15:293
218080399241,4cyclictest0-21swapper/312:20:283
218080199242,16cyclictest2184743-21ntpq07:20:291
2180801992416,5cyclictest2204665-21unixbench_multi08:25:381
2180800992418,4cyclictest2205303-21sort08:30:210
2180800992417,4cyclictest12450irq/33-eth008:05:300
218080399232,4cyclictest0-21swapper/311:25:283
218080399232,4cyclictest0-21swapper/308:25:213
218080399231,5cyclictest0-21swapper/311:52:413
218080399231,5cyclictest0-21swapper/310:40:333
218080399231,5cyclictest0-21swapper/309:15:273
218080399231,5cyclictest0-21swapper/307:15:263
218080399231,4cyclictest0-21swapper/312:15:313
218080399231,4cyclictest0-21swapper/312:05:283
218080399231,4cyclictest0-21swapper/310:50:333
218080399231,4cyclictest0-21swapper/310:35:263
218080399231,4cyclictest0-21swapper/310:25:343
218080399231,4cyclictest0-21swapper/307:40:243
218080399231,3cyclictest0-21swapper/311:40:323
218080399231,3cyclictest0-21swapper/311:35:293
218080299233,5cyclictest171rcu_preempt09:00:292
218080299232,16cyclictest2199724-21ntpq08:10:302
2180802992318,2cyclictest0-21swapper/207:10:022
2180802992314,6cyclictest2185294-21sh07:25:012
2180800992318,2cyclictest0-21swapper/009:00:370
218080399222,7cyclictest0-21swapper/311:00:293
218080399222,4cyclictest0-21swapper/310:05:303
218080399222,3cyclictest0-21swapper/308:20:333
218080399221,6cyclictest0-21swapper/312:10:323
218080399221,5cyclictest0-21swapper/312:30:343
218080399221,5cyclictest0-21swapper/311:45:323
218080399221,5cyclictest0-21swapper/311:05:193
218080399221,5cyclictest0-21swapper/309:35:333
218080399221,4cyclictest0-21swapper/312:29:133
218080399221,4cyclictest0-21swapper/311:10:223
218080399221,4cyclictest0-21swapper/310:55:273
218080399221,4cyclictest0-21swapper/310:20:323
218080399221,4cyclictest0-21swapper/309:25:283
218080399221,4cyclictest0-21swapper/309:15:003
218080399221,4cyclictest0-21swapper/308:55:313
218080399221,4cyclictest0-21swapper/308:55:313
218080399221,4cyclictest0-21swapper/308:50:293
218080399221,4cyclictest0-21swapper/308:40:263
218080399221,4cyclictest0-21swapper/307:50:273
218080399221,4cyclictest0-21swapper/307:45:303
218080399221,4cyclictest0-21swapper/307:45:303
218080399221,4cyclictest0-21swapper/307:35:273
218080399221,3cyclictest0-21swapper/310:30:303
218080399221,3cyclictest0-21swapper/310:30:303
218080299223,12cyclictest1054-21mta-sts-daemon08:50:322
2180801992215,4cyclictest2181082-21aten2.4-expect07:10:161
2180800992217,3cyclictest151ktimers/008:20:270
218080399211,5cyclictest0-21swapper/311:20:273
218080399211,5cyclictest0-21swapper/311:20:273
218080399211,5cyclictest0-21swapper/309:20:213
218080399211,4cyclictest0-21swapper/311:55:313
218080399211,4cyclictest0-21swapper/311:30:243
218080399211,4cyclictest0-21swapper/310:45:223
218080399211,4cyclictest0-21swapper/310:15:253
218080399211,4cyclictest0-21swapper/310:00:253
218080399211,4cyclictest0-21swapper/309:55:203
218080399211,4cyclictest0-21swapper/309:50:293
218080399211,4cyclictest0-21swapper/309:50:293
218080399211,4cyclictest0-21swapper/309:45:273
218080399211,4cyclictest0-21swapper/309:40:243
218080399211,4cyclictest0-21swapper/309:30:293
218080399211,4cyclictest0-21swapper/309:10:013
218080399211,4cyclictest0-21swapper/309:00:203
218080399211,4cyclictest0-21swapper/308:45:263
218080399211,4cyclictest0-21swapper/308:35:283
218080399211,4cyclictest0-21swapper/308:05:313
218080399211,4cyclictest0-21swapper/307:25:273
218080399211,4cyclictest0-21swapper/307:10:313
2180801992115,3cyclictest0-21swapper/107:10:021
2180800992115,3cyclictest0-21swapper/007:10:010
218080399201,5cyclictest0-21swapper/312:35:163
218080399201,5cyclictest0-21swapper/308:10:243
218080399201,4cyclictest0-21swapper/310:10:183
218080399201,4cyclictest0-21swapper/308:00:273
218080399201,4cyclictest0-21swapper/307:20:263
218080399201,3cyclictest0-21swapper/307:30:293
218080399101,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional