You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 07:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Thu May 14, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
54937222110,2sleep375-21kworker/3:1-events23:50:003
2938162740,5sleep30-21swapper/319:31:023
268279996651,13cyclictest569205-21apt-get00:10:011
5208682650,2sleep3471rcuc/323:20:593
3819292640,2sleep375-21kworker/3:1-events21:00:583
3819292640,2sleep375-21kworker/3:1-events21:00:583
268267996353,7cyclictest320138-21latency_hist20:00:000
268279996254,6cyclictest429341-21ntpq21:46:021
268267995847,9cyclictest439426-21ntpq21:56:020
268267995752,4cyclictest28265-21kworker/u8:1+flush-8:019:11:070
268279995537,15cyclictest549683-21apt-get23:50:521
268279995533,18cyclictest280-21systemd-journal00:01:221
268282995440,12cyclictest430231-21apt-get21:50:002
268279995445,7cyclictest289271-21ntpq19:26:051
268279995442,9cyclictest280-21systemd-journal20:30:221
268279995437,14cyclictest499091-21ntpq22:56:041
268267995449,3cyclictest519408-21kworker/u8:1+flush-8:023:26:110
268282995338,13cyclictest310477-21base6419:50:002
268267995348,3cyclictest519408-21kworker/u8:1+flush-8:000:21:090
268267995247,4cyclictest330528-21kworker/u8:1+flush-8:020:31:080
268267995247,4cyclictest28265-21kworker/u8:1+flush-8:019:50:570
268282995140,8cyclictest445726-21cp22:05:012
268279995143,5cyclictest504911-21kworker/u8:2+events_unbound23:21:061
268267995040,8cyclictest543548-21ntpq23:41:040
4956152490,1chrt495616-21/usr/sbin/munin22:55:532
268282994938,8cyclictest513859-21ntpq23:11:032
268279994936,10cyclictest485132-21latency_hist22:45:011
268279994930,17cyclictest280-21systemd-journal22:00:011
268282994837,8cyclictest290632-21diskstats19:30:542
268279994838,7cyclictest349215-21ntpq20:26:041
268267994836,9cyclictest375100-21base6420:55:000
268267994742,4cyclictest275470-21kworker/u8:2+flush-8:019:21:050
268282994629,3cyclictest319727-21unixbench_multi19:56:092
268279994627,16cyclictest474304-21ntpq22:31:041
268279994622,7cyclictest371331-21kthreadcore20:51:001
268267994641,4cyclictest519408-21kworker/u8:1+flush-8:000:01:050
268267994639,5cyclictest355472-21kworker/u8:0+flush-8:021:11:070
268267994637,7cyclictest538578-21ntpq23:36:010
268279994538,5cyclictest500263-21apt-get23:00:521
268279994538,5cyclictest500263-21apt-get23:00:521
268279994538,5cyclictest280-21systemd-journal00:39:321
268279994435,6cyclictest280-21systemd-journal21:21:231
268279994428,14cyclictest414364-21ntpq21:31:031
268279994423,19cyclictest369325-21ntpq20:46:071
268267994440,3cyclictest460018-21kworker/u8:3+flush-8:023:15:550
268267994440,3cyclictest330528-21kworker/u8:1+flush-8:022:41:120
268279994335,5cyclictest280-21systemd-journal22:21:211
268279994334,7cyclictest319231-21ntpq19:56:021
268267994333,7cyclictest569205-21apt-get00:10:010
268282994231,9cyclictest419492-21ntpq21:36:052
268282994222,8cyclictest598551-21ntpq00:36:072
268279994236,4cyclictest360543-21apt-get20:40:511
268279994235,4cyclictest268496-21cat19:10:011
268279994223,4cyclictest280-21systemd-journal19:43:211
268279994221,8cyclictest280-21systemd-journal21:55:001
268267994238,3cyclictest275470-21kworker/u8:2+flush-8:019:41:070
268267994238,3cyclictest275470-21kworker/u8:2+flush-8:019:15:560
268267994231,9cyclictest520282-21diskstats23:20:560
268267994231,8cyclictest569205-21apt-get00:09:590
268282994134,5cyclictest354999-21apt-get20:35:002
268282994124,4cyclictest334835-21unixbench_multi20:11:112
268279994132,7cyclictest275510-21/usr/sbin/munin19:16:031
268279994033,5cyclictest359306-21ntpq20:36:021
268279994020,4cyclictest880-21mta-sts-daemon22:06:001
268267994037,2cyclictest475424-21kworker/u8:0+events_unbound22:35:530
268267994030,8cyclictest446077-21diskstats22:05:540
268279993932,5cyclictest579616-21/usr/sbin/munin00:21:011
268279993931,6cyclictest325633-21/usr/sbin/munin20:06:041
268279993921,3cyclictest407-21mta-sts-daemon21:15:001
268267993935,3cyclictest460018-21kworker/u8:3+events_unbound23:10:540
268267993935,3cyclictest330528-21kworker/u8:1+events_unbound20:56:100
268267993935,3cyclictest275470-21kworker/u8:2+events_unbound19:36:080
268267993934,3cyclictest460018-21kworker/u8:3+flush-8:022:46:000
268267993933,4cyclictest290326-21kworker/u8:3+flush-8:020:06:120
268279993831,5cyclictest544747-21apt-get23:45:501
268279993827,8cyclictest454539-21ntpq22:11:041
268279993821,3cyclictest409061-21needreboot21:26:041
268279993819,2cyclictest0-21swapper/121:05:561
268279993817,3cyclictest0-21swapper/100:16:071
268267993834,3cyclictest559518-21kworker/u8:0+events_unbound00:36:090
268267993834,3cyclictest290326-21kworker/u8:3+events_unbound21:20:550
268267993832,4cyclictest559518-21kworker/u8:0+flush-8:000:31:060
268267993831,5cyclictest558368-21ntpq23:56:050
268267993829,7cyclictest460652-21/usr/sbin/munin22:21:030
268267993814,22cyclictest1-21systemd23:47:230
268279993732,4cyclictest275470-21kworker/u8:2+flush-8:020:00:531
268279993732,3cyclictest570469-21irqrtprio00:10:561
268279993730,5cyclictest310605-21apt-get19:50:521
268279993719,5cyclictest589486-21/usr/sbin/munin00:31:031
268279993718,3cyclictest280736-21cpuspeed_turbos19:20:531
268279993714,7cyclictest518810-21ntpq23:16:041
268267993735,1cyclictest290326-21kworker/u8:3+events_unbound21:40:520
268267993733,3cyclictest290326-21kworker/u8:3+events_unbound21:06:090
268267993732,4cyclictest330528-21kworker/u8:1+flush-8:020:26:090
268279993628,6cyclictest415755-21apt-get21:35:511
268279993626,8cyclictest459453-21ntpq22:16:031
268267993632,3cyclictest460018-21kworker/u8:3+events_unbound23:51:090
268267993632,3cyclictest290326-21kworker/u8:3+events_unbound19:31:080
268267993631,3cyclictest355472-21kworker/u8:0+events_unbound20:41:010
268267993631,3cyclictest330528-21kworker/u8:1+flush-8:022:11:010
268267993629,5cyclictest330528-21kworker/u8:1+flush-8:021:46:050
268282993519,4cyclictest371334-21kthreadcore20:50:592
268267993531,3cyclictest460018-21kworker/u8:3+events_unbound23:00:570
268267993531,3cyclictest460018-21kworker/u8:3+events_unbound23:00:560
268267993531,3cyclictest290326-21kworker/u8:3+flush-8:020:21:100
268267993528,5cyclictest305194-21apt-key19:45:010
268267993528,5cyclictest290326-21kworker/u8:3+flush-8:021:16:070
268279993425,6cyclictest280-21systemd-journal23:32:221
268279993423,9cyclictest588303-21ntpq00:26:021
268279993418,4cyclictest539815-21apt-get23:40:511
268267993430,3cyclictest504911-21kworker/u8:2+events_unbound23:31:070
268267993428,4cyclictest559518-21kworker/u8:0+events_unbound00:25:540
268267993330,2cyclictest330528-21kworker/u8:1+flush-8:021:36:110
268267993329,3cyclictest330528-21kworker/u8:1+events_unbound20:46:120
268267993327,5cyclictest460018-21kworker/u8:3+events_unbound23:06:070
268267993327,4cyclictest409092-21ntpq21:26:030
5925472320,5sleep3451irq_work/300:31:003
5506562320,2sleep3481ktimers/323:51:013
3440612320,3sleep10-21swapper/120:21:041
268282993225,5cyclictest280-21systemd-journal21:35:002
268279993224,6cyclictest280-21systemd-journal22:36:211
268267993228,3cyclictest330528-21kworker/u8:1+events_unbound21:31:060
268267993225,5cyclictest290326-21kworker/u8:3+events_unbound20:16:050
268267993223,7cyclictest267996-21latency_hist19:10:010
4830892310,6chrt0-21swapper/222:41:012
268282993123,6cyclictest280-21systemd-journal19:55:002
268279993126,4cyclictest330528-21kworker/u8:1+flush-8:022:01:531
268279993124,5cyclictest0-21swapper/122:25:581
268267993127,3cyclictest409706-21kworker/u8:0+events_unbound22:15:510
268267993025,4cyclictest465318-21kworker/u8:2+flush-8:022:31:060
268279992924,3cyclictest340238-21systemd-run20:19:591
268267992925,3cyclictest330528-21kworker/u8:1+flush-8:021:50:530
268267992925,3cyclictest238360-21kworker/u8:0+events_unbound19:25:540
268267992924,4cyclictest275470-21kworker/u8:2+events_unbound20:01:040
268267992924,3cyclictest465903-21fschecks_count22:25:550
268267992923,4cyclictest355472-21kworker/u8:0+events_unbound20:35:520
268282992821,5cyclictest280-21systemd-journal22:54:232
268279992823,2cyclictest0-21swapper/123:56:051
268267992824,3cyclictest460018-21kworker/u8:3+events_unbound22:50:540
268267992823,3cyclictest569665-21kworker/u8:3+events_unbound00:16:110
268282992723,2cyclictest344775-21unixbench_singl20:21:132
268282992719,6cyclictest470937-21cut22:30:552
268279992718,6cyclictest280-21systemd-journal21:01:231
268279992718,6cyclictest280-21systemd-journal21:01:221
268282992622,3cyclictest593756-21unixbench_multi00:31:112
268282992621,3cyclictest320926-21cpuspeed_turbos20:00:532
268279992622,3cyclictest519408-21kworker/u8:1+flush-8:023:25:531
268279992622,2cyclictest510525-21fschecks_time23:10:551
268279992621,3cyclictest295000-21cron19:35:001
268279992617,7cyclictest490374-21apt-get22:50:501
268267992622,3cyclictest290326-21kworker/u8:3+events_unbound21:04:230
268267992622,3cyclictest290326-21kworker/u8:3+events_unbound21:04:220
268282992521,3cyclictest563920-21unixbench_singl00:01:112
268282992510,4cyclictest0-21swapper/220:56:012
268279992520,3cyclictest305810-21cut19:45:551
268279992515,8cyclictest280-21systemd-journal23:10:001
268279992515,7cyclictest280-21systemd-journal22:46:231
268267992521,3cyclictest475424-21kworker/u8:0+events_unbound22:55:540
268267992521,3cyclictest330528-21kworker/u8:1+flush-8:022:01:040
28482422416,5sleep30-21swapper/319:21:283
268282992420,2cyclictest574927-21cpuspeed_turbos00:15:532
268282992420,2cyclictest450821-21aten2_r5power_e22:10:532
268282992420,2cyclictest268865-21fschecks_count19:10:552
268282992420,2cyclictest0-21swapper/221:26:012
268282992419,4cyclictest460349-21sendmail22:20:002
268279992419,3cyclictest539486-21idleruntime-cro23:40:001
268279992415,7cyclictest399123-21ntpq21:16:061
268282992319,2cyclictest365539-21aten2_r5power_p20:45:532
268282992315,4cyclictest476430-21kthreadcore22:35:592
26828299230,9cyclictest5323282kthreadcore23:30:592
268279992319,2cyclictest0-21swapper/120:55:561
268279992314,7cyclictest424399-21ntpq21:41:021
268267992317,4cyclictest0-21swapper/020:11:010
2643152232,14sleep1265130-21grep19:06:001
2811232220,6chrt281122-21irqcore19:20:582
26828299225,2cyclictest5247962sed23:25:012
268282992217,3cyclictest425327-21latency_hist21:45:012
268279992218,2cyclictest0-21swapper/119:35:541
268282992118,2cyclictest285539-21aten2_r5power_v19:25:532
268282992116,3cyclictest548966-21unixbench_multi23:46:102
268282992116,3cyclictest500486-21cpuspeed_turbos23:00:552
268282992116,3cyclictest500486-21cpuspeed_turbos23:00:552
268279992117,2cyclictest0-21swapper/120:11:061
268282992017,2cyclictest519711-21systemctl23:20:012
268282992016,3cyclictest436021-21fschecks_time21:55:542
268282992016,3cyclictest405013-21turbostat.cron21:25:012
268282992016,2cyclictest0-21swapper/219:35:582
268282992016,2cyclictest0-21swapper/200:10:012
268282992013,5cyclictest305015-21cp19:45:012
268282992011,7cyclictest551942-21tr23:51:022
2631412202,9sleep30-21swapper/319:05:023
4626392190,2chrt75-21kworker/3:1+events22:21:003
26828299192,2cyclictest0-21swapper/200:20:142
26828299192,12cyclictest505197-21apt-get23:05:522
268282991916,2cyclictest0-21swapper/219:15:592
268282991915,2cyclictest0-21swapper/200:25:552
268282991911,6cyclictest280-21systemd-journal22:50:002
268282991910,7cyclictest432370-21tr21:50:592
26828299182,11cyclictest465612-21apt-get22:25:522
268282991814,3cyclictest0-21swapper/220:06:132
268282991814,2cyclictest0-21swapper/223:41:012
268282991814,2cyclictest0-21swapper/221:16:122
268282991814,2cyclictest0-21swapper/220:40:002
268282991813,3cyclictest0-21swapper/221:00:542
268282991813,3cyclictest0-21swapper/221:00:532
2638002183,9sleep20-21swapper/219:05:572
26828299177,8cyclictest389470-21tr21:06:102
26828299172,11cyclictest364299-21ntpq20:41:052
268282991714,2cyclictest335605-21aten2_r5power_e20:15:522
268282991713,2cyclictest525317-21fschecks_time23:25:552
268282991713,2cyclictest0-21swapper/200:10:502
268282991712,3cyclictest0-21swapper/222:20:582
26828299171,12cyclictest347591-21ps20:26:002
26828299162,13cyclictest0-21swapper/223:55:042
268282991613,2cyclictest446094-21fschecks_count22:05:532
268282991612,3cyclictest0-21swapper/223:36:102
26828299161,11cyclictest390427-21apt-get21:10:512
4532792142,8sleep30-21swapper/322:11:023
4409062143,7sleep30-21swapper/322:00:553
5959152133,5sleep30-21swapper/300:36:003
5654692133,6sleep30-21swapper/300:05:583
5098502132,6sleep30-21swapper/323:10:013
4976852133,6sleep30-21swapper/322:56:003
4210882132,7sleep30-21swapper/321:40:563
3999992132,6sleep30-21swapper/321:20:013
3950342132,7sleep30-21swapper/321:15:003
3684682130,2sleep375-21kworker/3:1+events20:46:023
3541402132,7sleep30-21swapper/320:31:063
3144292132,6sleep30-21swapper/319:51:093
5813632122,6sleep30-21swapper/300:20:593
5582062122,6sleep30-21swapper/323:56:033
5441392122,6sleep30-21swapper/323:41:233
5288572122,6sleep30-21swapper/323:26:093
5147882122,5sleep30-21swapper/323:15:023
5041322122,6sleep30-21swapper/323:01:103
5041322122,6sleep30-21swapper/323:01:103
4946982122,6sleep30-21swapper/322:51:233
4854442122,6sleep30-21swapper/322:45:533
4716782122,6sleep30-21swapper/322:31:003
4663232123,5sleep30-21swapper/322:25:583
4566372122,6sleep30-21swapper/322:15:593
4462062122,6sleep30-21swapper/322:05:563
4352942122,6sleep30-21swapper/321:53:513
4296162122,6sleep30-21swapper/321:46:103
4154532122,5sleep30-21swapper/321:35:013
4110172122,6sleep30-21swapper/321:30:573
4007192122,6sleep30-21swapper/321:20:583
3858362122,6sleep30-21swapper/321:05:573
3709942122,6sleep30-21swapper/320:50:593
3609122122,6sleep30-21swapper/320:40:573
3601962120,6sleep30-21swapper/320:40:003
3454032123,5sleep30-21swapper/320:25:443
3443732122,6sleep30-21swapper/320:21:103
3260402122,6sleep30-21swapper/320:05:583
3057102122,6sleep30-21swapper/319:45:553
3000072122,5sleep30-21swapper/319:40:023
2956892122,6sleep30-21swapper/319:35:563
2759562122,6sleep30-21swapper/319:15:573
2689742122,6sleep30-21swapper/319:10:563
5845332112,6sleep30-21swapper/300:25:213
5744102112,4sleep30-21swapper/300:15:003
5696602112,5sleep30-21swapper/300:10:043
5633982112,5sleep30-21swapper/300:01:043
5394242112,6sleep30-21swapper/323:39:263
5302452112,5sleep30-21swapper/323:30:553
5139002112,5sleep30-21swapper/323:11:043
4845212112,5sleep30-21swapper/322:41:103
4752182112,5sleep30-21swapper/322:35:013
4058112112,5sleep30-21swapper/321:25:593
3754702112,5sleep30-21swapper/320:55:063
3394612112,5sleep30-21swapper/320:16:093
3308612112,5sleep30-21swapper/320:10:553
3245622112,5sleep30-21swapper/320:01:083
3166182112,5sleep30-21swapper/319:55:593
2854302112,5sleep30-21swapper/319:25:533
4356272102,5sleep30-21swapper/321:55:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional