You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 08:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Aug 22, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
370889521950,5sleep30-21swapper/319:22:463
37332382730,3sleep1281rcuc/119:47:441
37153962620,2chrt575-21kworker/3:2+events19:32:343
3693124995950,6cyclictest3744514-21latency_hist20:00:001
3693124995950,6cyclictest3744514-21latency_hist20:00:001
3693122995949,7cyclictest3849170-21/usr/sbin/munin21:47:520
3693122995746,9cyclictest275-21systemd-journal23:20:010
3693122995647,6cyclictest275-21systemd-journal23:55:000
3693122995549,4cyclictest3548738-21kworker/u8:4+flush-8:019:47:530
3693122995548,5cyclictest3838662-21kworker/u8:1+events_unbound23:10:010
3693122995449,3cyclictest3548737-21kworker/u8:2+flush-8:019:15:050
3693122995445,3cyclictest3878619-21kworker/u8:3+events_unbound23:05:010
3693122995342,4cyclictest3967186-21kworker/u8:0+events_unbound00:02:420
3693122995342,4cyclictest3967186-21kworker/u8:0+events_unbound00:02:420
3693122995247,3cyclictest3838662-21kworker/u8:1+flush-8:022:57:530
3693122995147,3cyclictest3838662-21kworker/u8:1+events_unbound22:02:510
3693122995146,4cyclictest3983109-21kworker/u8:4+events_unbound00:15:010
39142192500,3chrt575-21kworker/3:2+events22:52:423
3693122995046,3cyclictest3893049-21kworker/u8:0+events_unbound22:52:530
3693127994839,6cyclictest3834087-21cp21:30:012
3693124994837,8cyclictest275-21systemd-journal22:13:181
3693122994842,4cyclictest3798917-21kworker/u8:0+flush-8:021:07:520
3693122994741,4cyclictest3804200-21kworker/u8:3+events_unbound21:12:490
3693122994741,4cyclictest3747161-21kworker/u8:3+flush-8:020:42:470
3693122994641,3cyclictest3838662-21kworker/u8:1+events_unbound22:37:540
3693122994635,4cyclictest3846712-21kworker/u8:2+events_unbound22:15:000
3693122994440,3cyclictest3838662-21kworker/u8:1+events_unbound21:42:530
3693122994440,3cyclictest3769550-21kworker/u8:1+flush-8:020:28:190
3693122994439,4cyclictest3838662-21kworker/u8:1+events_unbound23:42:530
3693122994438,4cyclictest3937670-21kworker/u8:2+events_unbound23:47:480
3693124994325,7cyclictest4017911-21/usr/sbin/munin00:37:531
3693122994339,3cyclictest3878619-21kworker/u8:3+events_unbound22:17:390
3693122994339,3cyclictest3747161-21kworker/u8:3+flush-8:020:37:540
3693122994337,4cyclictest3804200-21kworker/u8:3+events_unbound21:32:370
3693124994233,7cyclictest275-21systemd-journal20:57:181
3693122994237,4cyclictest3967186-21kworker/u8:0+events_unbound00:35:330
3693122994237,4cyclictest3548738-21kworker/u8:4+events_unbound19:52:500
3693122994236,5cyclictest3983109-21kworker/u8:4+events_unbound00:10:000
3693127994132,7cyclictest3982800-21cp00:00:002
3693124994131,7cyclictest275-21systemd-journal22:41:191
3693122994137,3cyclictest3804200-21kworker/u8:3+flush-8:021:17:540
3693122994135,4cyclictest3769550-21kworker/u8:1+events_unbound21:02:380
3693122994133,6cyclictest275-21systemd-journal21:38:260
3693122994131,3cyclictest3744340-21kworker/u8:1+events_unbound20:07:370
36884742414,10sleep10-21swapper/119:07:201
3693124994032,6cyclictest3646672-21kworker/u8:0+flush-8:019:42:501
3693122994036,3cyclictest3982863-21kworker/u8:1+events_unbound00:27:390
3693122994036,3cyclictest3893049-21kworker/u8:0+flush-8:022:32:390
3693122994035,4cyclictest3548737-21kworker/u8:2+events_unbound20:52:380
3693122994035,3cyclictest3646672-21kworker/u8:0+events_unbound19:42:420
3693122994034,4cyclictest3769550-21kworker/u8:1+events_unbound20:57:500
3693124993931,6cyclictest3957313-21ntpq23:32:501
3693122993935,3cyclictest3548738-21kworker/u8:4+flush-8:019:28:200
3693122993935,3cyclictest3548737-21kworker/u8:2+events_unbound19:31:200
3693122993934,3cyclictest3937670-21kworker/u8:2+flush-8:023:40:000
3693122993929,8cyclictest3778460-21ntpq20:32:470
3693122993928,9cyclictest3982754-21munin-run00:00:000
3693124993832,4cyclictest3942907-21latency_hist23:20:011
3693124993827,8cyclictest3922191-21ntpq22:57:491
3693122993835,2cyclictest3838662-21kworker/u8:1+events_unbound22:47:380
3693122993834,3cyclictest3893049-21kworker/u8:0+flush-8:022:28:190
3693122993834,3cyclictest3646672-21kworker/u8:0+flush-8:019:22:540
3693122993833,4cyclictest3846712-21kworker/u8:2+flush-8:022:07:440
3693122993833,3cyclictest3983109-21kworker/u8:4+events_unbound00:17:370
3693122993833,3cyclictest3878619-21kworker/u8:3+flush-8:023:12:440
369312499373,18cyclictest0-21swapper/120:12:371
3693124993729,5cyclictest3749490-21latency_hist20:05:001
3693124993728,7cyclictest3704126-21ntpq19:17:491
3693122993733,3cyclictest3548737-21kworker/u8:2+flush-8:019:38:190
3693122993732,4cyclictest3548738-21kworker/u8:4+events_unbound19:12:480
3693122993732,4cyclictest3548737-21kworker/u8:2+events_unbound20:12:380
3693122993731,4cyclictest3846712-21kworker/u8:2+events_unbound21:57:440
3693122993728,7cyclictest275-21systemd-journal21:25:000
3693124993631,3cyclictest3929268-21irqrtprio23:07:421
3693124993630,4cyclictest3998889-21irqrtprio00:17:421
3693124993630,4cyclictest3809367-21apt-get21:07:331
3693124993617,9cyclictest0-21swapper/120:20:011
3693122993632,3cyclictest3893049-21kworker/u8:0+flush-8:022:42:400
3693122993632,3cyclictest3769550-21kworker/u8:1+events_unbound20:47:570
3693122993632,3cyclictest3747161-21kworker/u8:3+flush-8:020:17:520
3693122993631,3cyclictest3937670-21kworker/u8:2+events_unbound23:35:000
3693127993510,6cyclictest3779676-21apt-get20:37:362
3693122993530,4cyclictest3744340-21kworker/u8:1+events_unbound19:58:080
3693122993530,4cyclictest3744340-21kworker/u8:1+events_unbound19:58:080
3693122993530,3cyclictest3982863-21kworker/u8:1+events_unbound00:32:540
3693122993526,7cyclictest3938373-21/usr/sbin/munin23:17:470
3693124993428,4cyclictest3993546-21fschecks_time00:12:401
3693124993428,4cyclictest3843796-21basename21:40:001
3693124993416,9cyclictest0-21swapper/120:07:371
39629582330,6sleep20-21swapper/223:40:002
369312499330,21cyclictest573-21runrttasks00:02:521
369312499330,21cyclictest573-21runrttasks00:02:521
3693122993329,3cyclictest3846712-21kworker/u8:2+events_unbound21:53:020
3693122993329,3cyclictest3846712-21kworker/u8:2+events_unbound21:53:020
3693122993329,3cyclictest3798917-21kworker/u8:0+events_unbound21:25:020
3693122993328,3cyclictest3967186-21kworker/u8:0+events_unbound00:22:480
3693127993220,9cyclictest3698161-21cp19:15:012
3693124993227,3cyclictest3706023-21irqrtprio19:22:431
3693124993224,6cyclictest3947329-21ntpq23:22:491
3693124993220,9cyclictest3833667-21apt-get21:30:001
3693122993228,3cyclictest3747161-21kworker/u8:3+events_unbound20:20:020
369312499319,14cyclictest3858312-21cat21:52:541
369312499319,14cyclictest3858312-21cat21:52:541
369312499317,16cyclictest3786192-21cat20:42:411
3693124993126,4cyclictest3828698-21cron21:25:001
3693124993125,4cyclictest3799013-21cron20:55:001
3693124993125,4cyclictest3548737-21kworker/u8:2+flush-8:019:38:341
3693124993123,6cyclictest3868951-21apt-get22:07:351
3693124993122,7cyclictest3548738-21kworker/u8:4+flush-8:019:52:521
3693124993122,6cyclictest275-21systemd-journal22:38:181
3693124993120,2cyclictest161rcu_preempt21:32:501
3693127993021,7cyclictest3873435-21cron22:10:002
3693127993017,8cyclictest3834348-21apt-get21:32:352
3693127993016,9cyclictest3878799-21apt-get22:17:372
3693124993025,3cyclictest3963621-21cpuspeed_turbos23:42:371
3693124993022,6cyclictest3888288-21latency_hist22:25:011
3693124993022,6cyclictest3719165-21ntpq19:32:471
369312299302,7cyclictest0-21swapper/019:10:010
3693122993026,3cyclictest3947893-21kworker/u8:0+events_unbound23:27:190
3693122993025,3cyclictest3838662-21kworker/u8:1+events_unbound22:22:390
3693127992922,5cyclictest3714943-21cron19:30:002
3693127992915,9cyclictest3809367-21apt-get21:07:342
369312499295,12cyclictest30-21ksoftirqd/123:27:451
3693124992924,3cyclictest3864758-21irqrtprio22:02:411
3693124992923,4cyclictest3928112-21idleruntime-cro23:05:001
3693124992921,6cyclictest3783908-21unixbench_multi20:37:551
3693124992921,5cyclictest3977781-21cron23:55:001
3693124992915,2cyclictest0-21swapper/120:49:031
3693127992823,3cyclictest3760065-21cpuspeed_turbos20:17:372
3693127992819,7cyclictest3818794-21cat21:15:002
3693124992823,3cyclictest3992776-21sh00:10:001
3693124992823,3cyclictest0-21swapper/120:25:011
3693124992820,6cyclictest4003122-21apt-get00:22:331
369312499281,9cyclictest3912270-21ntpq22:47:461
3693122992824,3cyclictest3646672-21kworker/u8:0+events_unbound20:00:080
36928252283,9sleep20-21swapper/219:08:542
369312799279,3cyclictest3795418-21/usr/sbin/munin20:52:412
369312799277,4cyclictest3952352-21ntpq23:27:502
3693127992724,2cyclictest3709560-21unixbench_multi19:22:562
3693127992721,4cyclictest3853432-21unixbench_multi21:47:542
3693127992721,4cyclictest3754216-21unixbench_multi20:07:552
3693127992718,4cyclictest3983293-21apt-get00:02:352
3693127992718,4cyclictest3983293-21apt-get00:02:352
3693127992717,8cyclictest3992069-21ntpq00:07:492
369312499274,7cyclictest3714069-21cat19:27:451
369312499273,12cyclictest4012521-21cron00:30:011
3693124992722,3cyclictest1-21systemd23:38:511
3693124992721,4cyclictest3968322-21apt-get23:47:331
369312499272,13cyclictest3819250-21aten2_r5power_c21:17:361
3693124992721,3cyclictest0-21swapper/123:57:521
369312499272,11cyclictest0-21swapper/122:27:361
3693124992713,8cyclictest0-21swapper/121:42:391
369312799263,14cyclictest0-21swapper/219:47:452
3693127992621,3cyclictest3874174-21fschecks_count22:12:402
3693127992621,3cyclictest3739328-21unixbench_multi19:52:542
3693127992617,4cyclictest3789602-21apt-get20:47:362
3693127992617,4cyclictest3740046-21apt-get19:57:362
3693127992617,4cyclictest3740046-21apt-get19:57:362
3693127992617,4cyclictest3700371-21apt-get19:17:362
3693127992616,4cyclictest3719385-21python319:32:512
3693127992613,11cyclictest4016996-21/usr/sbin/munin00:32:532
3693127992612,11cyclictest3938741-21grep23:17:402
3693127992610,9cyclictest3725246-21apt-get19:42:352
369312499267,5cyclictest30-21ksoftirqd/120:32:441
3693124992622,2cyclictest3937961-21idleruntime-cro23:15:001
3693124992615,2cyclictest0-21swapper/122:52:511
369312799256,7cyclictest0-21swapper/200:16:252
369312799255,5cyclictest0-21swapper/222:33:082
369312799253,5cyclictest3993232-21apt-get00:12:362
3693127992521,2cyclictest3973339-21aten2_r5power_e23:52:362
3693127992521,2cyclictest3804725-21fschecks_time21:02:392
3693127992516,4cyclictest4007139-21python300:22:522
3693127992516,4cyclictest3963388-21apt-get23:42:362
3693127992516,4cyclictest3913470-21apt-get22:52:362
369312499256,6cyclictest30-21ksoftirqd/122:32:521
369312499255,12cyclictest30-21ksoftirqd/121:46:591
369312499255,11cyclictest30-21ksoftirqd/122:17:441
369312499254,13cyclictest30-21ksoftirqd/121:58:341
369312499254,12cyclictest30-21ksoftirqd/120:27:491
369312499254,12cyclictest30-21ksoftirqd/100:27:581
3693124992520,2cyclictest0-21swapper/119:12:381
3693124992519,1cyclictest161rcu_preempt21:01:361
369312799247,7cyclictest0-21swapper/220:55:282
369312799246,8cyclictest0-21swapper/221:52:342
369312799246,8cyclictest0-21swapper/221:52:342
369312799245,7cyclictest0-21swapper/223:07:392
369312799245,4cyclictest3758505-21memory20:12:482
369312799243,8cyclictest3947663-21find23:22:552
3693127992416,3cyclictest3784629-21apt-get20:42:342
3693127992412,7cyclictest3724820-21/usr/sbin/munin19:40:002
369312499240,11cyclictest3818137-21ntpq21:12:501
369312799238,7cyclictest0-21swapper/223:01:162
369312799236,7cyclictest0-21swapper/220:02:192
369312799234,7cyclictest0-21swapper/222:00:332
3693127992314,4cyclictest3778468-21ntpq20:32:472
3693127992313,8cyclictest3765539-21/usr/sbin/munin20:22:442
3693127992312,7cyclictest3919846-21kthreadcore22:57:432
3693127992310,10cyclictest3859346-21/usr/sbin/munin21:57:392
369312799226,7cyclictest0-21swapper/223:12:032
369312799226,6cyclictest0-21swapper/220:25:552
369312799225,7cyclictest0-21swapper/222:46:002
369312799225,7cyclictest0-21swapper/222:40:342
369312799224,8cyclictest0-21swapper/221:42:412
3693127992212,8cyclictest3820139-21/usr/sbin/munin21:17:422
3693127992211,9cyclictest4011699-21/usr/sbin/munin00:27:462
3693127992210,9cyclictest3968341-21cut23:47:352
3693127992210,9cyclictest3888436-21latency_hist22:25:012
38375992210,4chrt0-21swapper/321:32:443
369312799218,10cyclictest3823994-21chrt21:20:312
369312799215,7cyclictest0-21swapper/222:35:362
369312799215,7cyclictest0-21swapper/221:35:252
369312799215,7cyclictest0-21swapper/200:35:352
3693127992112,7cyclictest3954394-21ps23:32:422
3693127992111,8cyclictest3889240-21/usr/sbin/munin22:27:422
36880982193,7sleep30-21swapper/319:05:013
37504142154,7sleep30-21swapper/320:07:423
38279902142,8sleep30-21swapper/321:22:503
36971672144,6sleep30-21swapper/319:12:483
39444932133,6sleep30-21swapper/323:22:443
39296542132,6sleep30-21swapper/323:07:443
39198432133,6sleep30-21swapper/322:57:443
38129282132,7sleep30-21swapper/321:07:453
37892152132,6sleep30-21swapper/320:45:023
37832622132,7sleep30-21swapper/320:37:473
37644052132,6sleep30-21swapper/320:20:023
40076062120,8sleep30-21swapper/300:25:003
40009812122,7sleep30-21swapper/300:17:463
39945052122,6sleep30-21swapper/300:12:453
39837872123,5sleep30-21swapper/300:02:413
39837872123,5sleep30-21swapper/300:02:413
39737912122,6sleep30-21swapper/323:52:423
39538482122,6sleep30-21swapper/323:32:403
39480952122,5sleep30-21swapper/323:25:013
39388922121,7sleep30-21swapper/323:17:433
39229302122,5sleep30-21swapper/323:00:013
39035562122,6sleep30-21swapper/322:42:193
38991742123,6sleep30-21swapper/322:37:423
38940462122,6sleep30-21swapper/322:32:403
38883332122,6sleep30-21swapper/322:25:023
38742622122,6sleep30-21swapper/322:12:423
38578722122,6sleep30-21swapper/321:52:483
38578722122,6sleep30-21swapper/321:52:483
38494342123,6sleep30-21swapper/321:47:383
38479572122,6sleep30-21swapper/321:42:483
38336552122,7sleep30-21swapper/321:30:003
38196212122,6sleep30-21swapper/321:17:403
38140252122,5sleep30-21swapper/321:10:023
38082742123,5sleep30-21swapper/321:02:523
37994732122,6sleep30-21swapper/320:57:383
37982992122,6sleep30-21swapper/320:52:483
37748392122,6sleep30-21swapper/320:32:383
37603042123,6sleep30-21swapper/320:17:413
37550972122,6sleep30-21swapper/320:12:393
37452542122,6sleep30-21swapper/320:02:403
37344812122,6sleep30-21swapper/319:47:563
37293472122,6sleep30-21swapper/319:42:543
37245432122,6sleep30-21swapper/319:37:563
36980132122,5sleep30-21swapper/319:15:013
40218052112,5sleep30-21swapper/300:37:533
40126112112,5sleep30-21swapper/300:30:023
40085022112,5sleep30-21swapper/300:27:413
39884962112,5sleep30-21swapper/300:07:393
39780362112,5sleep30-21swapper/323:55:193
39684082112,6sleep30-21swapper/323:47:363
39631662112,5sleep30-21swapper/323:40:023
39584162112,6sleep30-21swapper/323:37:373
39332612113,5sleep30-21swapper/323:10:203
39085092112,5sleep30-21swapper/322:47:333
38840592112,5sleep30-21swapper/322:22:413
38790432112,6sleep30-21swapper/322:17:403
38688892112,5sleep30-21swapper/322:07:083
38639312112,5sleep30-21swapper/322:01:313
38590912112,5sleep30-21swapper/321:57:373
38390622112,5sleep30-21swapper/321:35:023
37848772112,6sleep30-21swapper/320:42:373
37696372112,6sleep30-21swapper/320:26:203
37398512112,6sleep30-21swapper/319:55:223
37398512112,6sleep30-21swapper/319:55:223
37355372112,5sleep30-21swapper/319:52:413
37105772112,5sleep30-21swapper/319:27:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional