You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 19:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sun Jan 11, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1522299250,1cyclictest29743-21apt-get08:46:442
15226992423,1cyclictest31437-21apt-get09:56:443
1522299240,23cyclictest5845-21apt-get10:10:012
1522099240,23cyclictest0-21swapper/110:26:441
1522099240,23cyclictest0-21swapper/107:56:431
1522099240,23cyclictest0-21swapper/107:26:441
15216992422,1cyclictest23825-21awk09:40:000
1521699240,23cyclictest29718-21awk07:40:010
15226992321,1cyclictest12148-21awk08:10:013
15226992320,2cyclictest606-21dbus-daemon09:50:013
1522699230,22cyclictest0-21swapper/310:46:453
1522299230,2cyclictest22390-21sh07:25:012
1522299230,22cyclictest7302-21grep08:00:002
1522299230,22cyclictest606-21dbus-daemon10:00:012
1522299230,1cyclictest28625-21cron09:50:012
1522099230,22cyclictest26518-21cron08:40:001
1522099230,21cyclictest20011-21awk07:20:011
15216992323,0cyclictest10663-21perf12:35:010
1521699230,23cyclictest0-21swapper/008:11:450
1522699220,21cyclictest5270-21apt-get07:56:433
1522699220,21cyclictest20757-21apt-get11:46:453
1522699220,21cyclictest0-21swapper/312:20:013
1522699220,21cyclictest0-21swapper/312:20:013
1522699220,21cyclictest0-21swapper/311:21:443
1522699220,21cyclictest0-21swapper/310:31:443
1522699220,21cyclictest0-21swapper/310:16:433
1522699220,21cyclictest0-21swapper/309:21:443
1522699220,21cyclictest0-21swapper/308:15:013
1522699220,1cyclictest17945-21idleruntime-cro11:40:013
15222992220,1cyclictest27570-21sendmail-msp12:00:002
1522299220,21cyclictest428-21apt-get07:46:442
1522299220,21cyclictest24537-21apt-get08:36:442
1522299220,21cyclictest0-21swapper/208:06:442
1522299220,1cyclictest4860-21cron07:55:002
1522299220,0cyclictest1737-21apt-get08:55:052
15220992222,0cyclictest30609-21apt-get12:10:021
15220992220,1cyclictest10764-21latency_hist11:25:011
15220992220,1cyclictest0-21swapper/112:10:001
1522099220,21cyclictest8329-21sendmail11:20:001
1522099220,21cyclictest16997-21apt-get09:26:431
1522099220,21cyclictest1434-21apt-get10:01:451
1522099220,1cyclictest29042-21apt-get09:51:441
1522099220,1cyclictest28230-21munin-run12:05:001
15216992221,1cyclictest21108-21apt-get10:41:440
15216992220,1cyclictest0-21swapper/010:55:010
1521699220,21cyclictest8363-21awk11:20:000
1521699220,21cyclictest32137-21apt-get08:51:440
1521699220,21cyclictest26563-21sendmail-msp08:40:000
1521699220,21cyclictest0-21swapper/010:16:430
1521699220,1cyclictest11143-21apt-get11:26:440
1521699220,1cyclictest1052-21munin-run10:00:010
15226992120,1cyclictest5857-21perf10:10:013
15226992120,1cyclictest22783-21apt-get07:26:443
15226992120,1cyclictest18362-21apt-get11:41:453
15226992120,1cyclictest0-21swapper/310:36:453
15226992119,1cyclictest2358-21apt-get08:55:053
1522699210,21cyclictest0-21swapper/308:21:453
1522699210,20cyclictest0-21swapper/307:51:443
1522699210,20cyclictest0-21swapper/307:16:443
1522699210,0cyclictest25545-21apt-get11:56:433
15222992121,0cyclictest28295-21apt-get10:56:442
15222992120,1cyclictest6955-21munin-run09:05:012
15222992120,0cyclictest0-21swapper/210:46:452
15222992119,1cyclictest0-21swapper/211:46:452
1522299210,0cyclictest0-21swapper/212:16:442
1522299210,0cyclictest0-21swapper/212:16:442
1522299210,0cyclictest0-21swapper/211:41:452
1522299210,0cyclictest0-21swapper/211:31:432
1522299210,0cyclictest0-21swapper/211:06:442
1522299210,0cyclictest0-21swapper/209:51:442
15220992121,0cyclictest6631-21apt-get10:11:461
15220992121,0cyclictest0-21swapper/110:21:441
15220992120,1cyclictest4937-21apt-get09:01:441
15220992120,1cyclictest1369-21apt-get12:16:441
15220992120,1cyclictest1369-21apt-get12:16:441
15220992119,1cyclictest24145-21perf08:35:001
1522099210,21cyclictest0-21swapper/109:16:441
1522099210,20cyclictest13146-21perf11:30:011
1522099210,20cyclictest0-21swapper/107:51:431
1522099210,1cyclictest15944-21apt-get11:36:441
1522099210,0cyclictest0-21swapper/109:11:421
1522099210,0cyclictest0-21swapper/108:55:051
1522099210,0cyclictest0-21swapper/107:31:441
15216992121,0cyclictest24257-21apt-get09:41:460
15216992121,0cyclictest0-21swapper/010:46:450
1521699210,21cyclictest0-21swapper/010:31:450
1521699210,20cyclictest1079-21apt-get11:06:440
1521699210,20cyclictest0-21swapper/007:56:430
1521699210,0cyclictest0-21swapper/009:51:430
15226992020,0cyclictest0-21swapper/311:31:433
15226992020,0cyclictest0-21swapper/309:06:453
15226992020,0cyclictest0-21swapper/308:36:443
15226992020,0cyclictest0-21swapper/308:31:443
1522699200,0cyclictest0-21swapper/310:56:443
1522699200,0cyclictest0-21swapper/309:36:443
1522699200,0cyclictest0-21swapper/309:31:453
1522699200,0cyclictest0-21swapper/309:01:443
15222992020,0cyclictest19745-21apt-get08:26:432
15222992020,0cyclictest0-21swapper/211:21:432
15222992020,0cyclictest0-21swapper/208:01:452
15222992019,1cyclictest17353-21apt-get08:21:442
15222992019,1cyclictest13892-21apt-get10:26:442
15222992018,1cyclictest24140-21grep08:35:002
15222992018,1cyclictest16975-21munin-run08:20:012
1522299200,20cyclictest0-21swapper/207:36:432
1522299200,19cyclictest0-21swapper/211:51:442
1522299200,0cyclictest0-21swapper/212:06:412
1522299200,0cyclictest0-21swapper/209:26:432
15220992020,0cyclictest28956-21apt-get08:45:021
15220992020,0cyclictest0-21swapper/110:51:451
15220992020,0cyclictest0-21swapper/109:36:431
15220992020,0cyclictest0-21swapper/109:06:451
15220992020,0cyclictest0-21swapper/107:46:441
15220992019,1cyclictest23504-21apt-get10:46:451
15220992018,1cyclictest8260-21munin-run12:30:011
15220992018,1cyclictest22753-21basename11:50:011
1522099200,1cyclictest619-21polkitd11:00:011
1522099200,19cyclictest9076-21apt-get10:16:431
1522099200,19cyclictest15577-21apt-get07:11:441
1522099200,0cyclictest0-21swapper/108:21:441
15216992020,0cyclictest23151-21apt-get11:51:440
15216992020,0cyclictest0-21swapper/012:16:440
15216992020,0cyclictest0-21swapper/012:16:440
15216992020,0cyclictest0-21swapper/011:56:420
15216992020,0cyclictest0-21swapper/011:21:440
15216992020,0cyclictest0-21swapper/010:56:440
15216992020,0cyclictest0-21swapper/008:31:440
15216992019,1cyclictest3715-21apt-get11:10:020
15216992018,1cyclictest5794-21sh12:25:010
1521699200,19cyclictest0-21swapper/007:35:010
1521699200,0cyclictest18692-21apt-get10:36:450
1521699200,0cyclictest0-21swapper/012:36:450
1521699200,0cyclictest0-21swapper/011:01:450
1521699200,0cyclictest0-21swapper/009:46:430
15226991919,0cyclictest6275-21apt-get11:16:433
15226991919,0cyclictest0-21swapper/312:26:443
15226991919,0cyclictest0-21swapper/312:21:453
15226991919,0cyclictest0-21swapper/312:11:453
15226991919,0cyclictest0-21swapper/311:51:443
15226991919,0cyclictest0-21swapper/311:06:443
15226991919,0cyclictest0-21swapper/308:45:023
15226991919,0cyclictest0-21swapper/307:46:443
15222991919,0cyclictest0-21swapper/212:10:022
15222991917,1cyclictest14538-21apt08:15:012
15222991917,1cyclictest11143-21apt-get11:26:442
15222991917,1cyclictest0-21swapper/210:16:422
1522299190,0cyclictest0-21swapper/211:10:022
1522299190,0cyclictest0-21swapper/210:31:442
1522299190,0cyclictest0-21swapper/207:16:442
15220991919,0cyclictest0-21swapper/109:41:461
15220991919,0cyclictest0-21swapper/108:01:441
15220991919,0cyclictest0-21swapper/107:21:461
15220991918,0cyclictest0-21swapper/111:41:451
15220991918,0cyclictest0-21swapper/110:41:441
15216991919,0cyclictest0-21swapper/010:21:430
15216991919,0cyclictest0-21swapper/008:45:020
15216991919,0cyclictest0-21swapper/007:21:460
15216991919,0cyclictest0-21swapper/007:16:440
15216991918,1cyclictest28227-21perf12:05:000
15216991917,1cyclictest993-21munin-run12:15:010
15216991917,1cyclictest3840-21apt-get10:06:430
15216991917,1cyclictest14213-21munin-run09:20:010
15226991818,0cyclictest0-21swapper/310:51:443
15226991818,0cyclictest0-21swapper/310:41:443
15226991818,0cyclictest0-21swapper/310:10:023
15226991818,0cyclictest0-21swapper/308:41:443
15226991818,0cyclictest0-21swapper/308:16:443
15226991818,0cyclictest0-21swapper/307:41:443
15226991817,1cyclictest20392-21apt-get07:21:463
15226991816,1cyclictest0-21swapper/310:01:453
15222991818,0cyclictest8645-21apt-get12:31:452
15222991818,0cyclictest0-21swapper/210:41:442
15222991818,0cyclictest0-21swapper/209:36:442
15222991818,0cyclictest0-21swapper/209:11:422
15222991818,0cyclictest0-21swapper/207:26:442
15222991817,1cyclictest24118-21apt-get09:40:492
15222991817,1cyclictest0-21swapper/207:41:442
15222991816,1cyclictest619-21polkitd12:30:012
1522299180,17cyclictest605-21avahi-daemon09:17:402
15220991818,0cyclictest3803-21apt-get12:21:451
15220991817,0cyclictest26953-21apt-get08:41:441
15220991817,0cyclictest19395-21apt-get09:31:451
15220991816,2cyclictest13539-21apt-get11:31:431
15220991815,2cyclictest32093-21dump-pmu-power07:45:001
15216991818,0cyclictest0-21swapper/009:26:430
15216991818,0cyclictest0-21swapper/009:01:440
15216991818,0cyclictest0-21swapper/008:41:440
15216991818,0cyclictest0-21swapper/008:01:440
15216991816,1cyclictest21755-21latency_hist08:30:010
1521699180,17cyclictest0-21swapper/011:46:510
15226991717,0cyclictest0-21swapper/311:11:453
15226991717,0cyclictest0-21swapper/308:01:443
15222991717,0cyclictest0-21swapper/211:36:452
15222991717,0cyclictest0-21swapper/210:51:442
15222991715,1cyclictest16603-21awk09:25:002
15220991717,0cyclictest14926-21apt-get08:16:441
15220991717,0cyclictest0-21swapper/111:51:441
15220991717,0cyclictest0-21swapper/111:00:021
15220991717,0cyclictest0-21swapper/109:56:441
15220991717,0cyclictest0-21swapper/108:11:451
1522099170,0cyclictest0-21swapper/111:11:451
15216991717,0cyclictest30094-21apt-get07:41:430
15216991717,0cyclictest0-21swapper/011:41:450
15216991717,0cyclictest0-21swapper/007:51:440
15216991716,1cyclictest22787-21chrt07:26:440
15216991716,0cyclictest0-21swapper/011:36:450
15226991616,0cyclictest27677-21apt-get07:36:433
15226991616,0cyclictest0-21swapper/311:00:033
15226991616,0cyclictest0-21swapper/307:11:443
15226991614,1cyclictest0-21swapper/312:05:003
15222991616,0cyclictest0-21swapper/210:36:452
15222991616,0cyclictest0-21swapper/209:06:452
1522299160,7cyclictest0-21swapper/210:11:462
15216991616,0cyclictest28601-21apt-get12:06:410
15216991616,0cyclictest0-21swapper/008:56:440
15216991616,0cyclictest0-21swapper/008:21:450
15216991616,0cyclictest0-21swapper/008:16:440
15216991616,0cyclictest0-21swapper/008:06:440
15216991614,1cyclictest3463-21munin-run10:05:010
15226991515,0cyclictest0-21swapper/309:16:443
1522699150,14cyclictest29242-21hddtemp_smartct12:06:443
15222991515,0cyclictest0-21swapper/212:36:442
15222991515,0cyclictest0-21swapper/211:01:442
15222991515,0cyclictest0-21swapper/209:31:452
15222991513,1cyclictest0-21swapper/208:43:592
15220991515,0cyclictest0-21swapper/112:31:451
15220991514,1cyclictest0-21swapper/110:06:431
1522099150,1cyclictest0-21swapper/107:40:011
15216991515,0cyclictest0-21swapper/010:11:460
15216991512,2cyclictest611-21irqbalance10:28:010
1521699150,1cyclictest14328-21/usr/sbin/munin11:31:470
1521699150,0cyclictest428-21apt-get07:46:440
1522699140,14cyclictest0-21swapper/312:31:423
1522699140,14cyclictest0-21swapper/311:25:103
1522699140,14cyclictest0-21swapper/310:21:113
1522699140,14cyclictest0-21swapper/309:50:443
1522699140,14cyclictest0-21swapper/309:41:133
1522699140,14cyclictest0-21swapper/309:10:013
1522699140,14cyclictest0-21swapper/307:30:553
1522699140,0cyclictest0-21swapper/309:25:313
1522699140,0cyclictest0-21swapper/308:50:593
1522699140,0cyclictest0-21swapper/308:25:573
15222991414,0cyclictest0-21swapper/212:20:472
15222991414,0cyclictest0-21swapper/210:01:032
1522299140,14cyclictest0-21swapper/211:15:492
1522299140,14cyclictest0-21swapper/210:21:382
1522299140,14cyclictest0-21swapper/208:50:112
1522299140,14cyclictest0-21swapper/207:30:212
1522299140,14cyclictest0-21swapper/207:11:362
1522299140,0cyclictest0-21swapper/212:00:062
15220991414,0cyclictest0-21swapper/111:56:101
1522099140,14cyclictest0-21swapper/112:35:391
15216991414,0cyclictest0-21swapper/009:20:030
15216991414,0cyclictest0-21swapper/009:10:060
1521699140,14cyclictest0-21swapper/012:25:190
1521699140,14cyclictest0-21swapper/009:30:040
1521699140,14cyclictest0-21swapper/007:11:110
1521699140,0cyclictest0-21swapper/009:05:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional