You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 07:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sat Jun 06, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5662921840,2sleep3481ktimers/323:55:013
3965376996153,6cyclictest36528-21/usr/sbin/munin23:35:501
3965379995751,4cyclictest4119222-21kworker/u8:1+events_unbound23:00:452
3965379995652,3cyclictest4056833-21kworker/u8:1+events_unbound21:10:512
3965372995441,10cyclictest4171419-21diskstats22:35:370
3965372995343,7cyclictest4055918-21ntpq20:35:450
3965376995140,8cyclictest1704890-21systemd-journal20:00:021
41677442490,5chrt0-21swapper/222:30:412
3965376994942,6cyclictest3960308-21kworker/u8:3+flush-8:019:20:451
3965376994842,5cyclictest4086787-21kworker/u8:0+flush-8:021:15:471
3965376994839,7cyclictest4065853-21ntpq20:45:471
3965376994839,7cyclictest4050931-21ntpq20:30:451
3965372994736,8cyclictest4067255-21diskstats20:50:340
3965379994641,4cyclictest4119222-21kworker/u8:1+events_unbound22:20:492
3965376994640,4cyclictest4006813-21kworker/u8:2+flush-8:020:05:371
3965376994636,8cyclictest4030936-21ntpq20:10:461
3965376994535,8cyclictest35349-21ntpq23:30:441
3965372994440,3cyclictest61562-21kworker/u8:3+events_unbound00:10:510
3965376994235,4cyclictest4006813-21kworker/u8:2+flush-8:019:55:371
3965372994236,4cyclictest4032099-21apt-get20:15:330
40156872410,5sleep3451irq_work/319:55:433
3965379994137,3cyclictest4119222-21kworker/u8:1+events_unbound21:50:002
3965376994136,3cyclictest11513-21kworker/u8:2+events_unbound00:05:331
3965376994131,8cyclictest4161250-21/usr/sbin/munin22:25:501
3965376994034,4cyclictest4006813-21kworker/u8:2+flush-8:019:50:501
396537299402,37cyclictest4086563-21kworker/0:0+events00:18:530
3965379993935,3cyclictest3933392-21kworker/u8:1+flush-8:019:15:502
3965379993929,8cyclictest46510-21apt-get23:45:012
3965376993934,3cyclictest96433-21apt-get00:35:321
3965379993834,3cyclictest11513-21kworker/u8:2+flush-8:023:45:362
3965376993830,6cyclictest1704890-21systemd-journal22:15:491
3965376993830,6cyclictest1704890-21systemd-journal20:21:491
3965376993828,8cyclictest4121707-21diskstats21:45:351
3965376993731,4cyclictest80478-21ntpq00:15:471
3965372993729,6cyclictest4051828-21latency_hist20:35:010
3965376993632,3cyclictest4036755-21munin-run20:20:011
3965376993631,3cyclictest4116523-21apt-get21:40:301
3965376993628,6cyclictest1912-21apt-get23:00:321
3965376993530,3cyclictest4119222-21kworker/u8:1+flush-8:021:55:421
3965376993529,4cyclictest4006813-21kworker/u8:2+flush-8:020:55:481
3965376993529,4cyclictest3945595-21kworker/u8:0+flush-8:019:45:361
3965376993528,6cyclictest4119222-21kworker/u8:1+flush-8:022:40:481
3965376993527,6cyclictest90823-21grep00:25:501
396537299352,31cyclictest4086563-21kworker/0:0+events21:43:210
396537299352,28cyclictest3970448-21kworker/0:0+events19:40:130
396537299352,28cyclictest3970448-21kworker/0:0+events19:40:130
396490323521,9sleep10-21swapper/119:06:301
69382340,2sleep3471rcuc/323:05:333
69382340,2sleep3471rcuc/323:05:323
3965379993430,3cyclictest4051585-21kworker/u8:0+events_unbound20:45:432
3965376993429,4cyclictest4006813-21kworker/u8:2+flush-8:021:25:431
3965376993429,4cyclictest4006813-21kworker/u8:2+flush-8:021:25:421
3965376993428,5cyclictest3933392-21kworker/u8:1+flush-8:019:30:471
3965376993426,6cyclictest4181057-21apt-get22:45:311
3965376993423,8cyclictest4057068-21/usr/sbin/munin20:40:451
3965372993423,8cyclictest41760-21munin-node23:40:300
396537299342,31cyclictest4086563-21kworker/0:0+events22:33:050
396537299342,31cyclictest4051571-21kworker/0:2+events20:47:190
3965376993328,4cyclictest4101408-21kworker/u8:3+flush-8:023:20:481
3965376993327,4cyclictest4140372-21kworker/u8:0+flush-8:023:05:501
3965376993327,4cyclictest4140372-21kworker/u8:0+flush-8:023:05:491
396537299339,19cyclictest4086563-21kworker/0:0+events22:45:320
396537299332,30cyclictest4086563-21kworker/0:0+events22:57:570
396537299332,30cyclictest4086563-21kworker/0:0+events22:08:130
396537299332,30cyclictest3970448-21kworker/0:0+events19:18:380
396537299332,27cyclictest3960313-21kworker/0:2+events19:10:190
396537299331,31cyclictest4086563-21kworker/0:0+events22:14:300
396537299331,30cyclictest1-21systemd21:55:010
396537299331,26cyclictest4086563-21kworker/0:0+events21:55:020
3965372993311,17cyclictest3970448-21kworker/0:0+events20:10:070
3965376993227,4cyclictest3960308-21kworker/u8:3+flush-8:019:10:481
3965376993227,3cyclictest4112289-21irqrtprio21:35:401
3965376993223,7cyclictest4130169-21ntpq21:50:471
3965376993220,9cyclictest1704890-21systemd-journal21:31:141
3965376993218,4cyclictest421rcuog/220:35:441
3965372993223,7cyclictest4005892-21ntpq19:45:430
396537299322,29cyclictest3970448-21kworker/0:0+events19:38:320
3965372993221,9cyclictest0-21swapper/020:00:010
396537299321,30cyclictest4086563-21kworker/0:0+events22:26:460
396537299321,26cyclictest4086563-21kworker/0:0+events23:10:280
3965376993126,3cyclictest4140819-21unixbench_multi22:00:521
3965376993123,6cyclictest4140372-21kworker/u8:0+events_unbound23:10:481
3965376993120,9cyclictest95322-21ntpq00:30:481
3965376993120,8cyclictest85365-21ntpq00:20:451
396537299318,18cyclictest3970448-21kworker/0:0+events19:25:290
3965372993123,6cyclictest4126155-21latency_hist21:50:010
396537299312,28cyclictest3970448-21kworker/0:0+events20:10:010
396537299312,25cyclictest4086563-21kworker/0:0+events23:05:170
396537299312,25cyclictest4086563-21kworker/0:0+events23:05:170
396537299311,29cyclictest4086563-21kworker/0:0+events23:35:130
396537299311,25cyclictest4086563-21kworker/0:0+events00:30:070
396537299310,29cyclictest4086563-21kworker/0:0+events21:18:290
3965379993025,4cyclictest4119222-21kworker/u8:1+events_unbound22:51:062
3965376993026,2cyclictest4042963-21irqrtprio20:25:391
3965376993025,3cyclictest21482-21systemd-run23:20:001
3965376993024,3cyclictest4006813-21kworker/u8:2+flush-8:021:05:451
3965376993022,6cyclictest21486-21kworker/u8:1+flush-8:023:40:461
3965376993022,6cyclictest21486-21kworker/u8:1+events_unbound23:55:471
396537299305,20cyclictest3970448-21kworker/0:0+events19:50:170
396537299304,21cyclictest4086563-21kworker/0:0+events21:20:130
396537299302,27cyclictest4086563-21kworker/0:0+events23:01:020
396537299302,27cyclictest4086563-21kworker/0:0+events22:42:570
396537299302,27cyclictest4076657-21kworker/0:1+events21:09:170
396537299302,23cyclictest4051571-21kworker/0:2+events21:00:250
3965372993010,16cyclictest4086563-21kworker/0:0+events23:15:010
40945972290,6sleep30-21swapper/321:15:423
3965379992925,3cyclictest21486-21kworker/u8:1+events_unbound23:30:542
3965379992924,4cyclictest11513-21kworker/u8:2+events_unbound00:35:152
3965379992924,3cyclictest4101408-21kworker/u8:3+flush-8:023:10:012
3965376992925,3cyclictest3933392-21kworker/u8:1+flush-8:019:15:501
3965376992924,3cyclictest4091521-21systemd-run21:15:001
3965376992920,6cyclictest1704890-21systemd-journal00:04:031
3965376992917,9cyclictest4169994-21ntpq22:30:451
396537299298,20cyclictest4051571-21kworker/0:2+events20:44:270
396537299296,19cyclictest4086563-21kworker/0:0+events21:10:260
396537299296,18cyclictest4086563-21kworker/0:0+events21:30:170
396537299296,18cyclictest4086563-21kworker/0:0+events00:35:080
396537299296,18cyclictest4086563-21kworker/0:0+events00:05:070
396537299294,20cyclictest4086563-21kworker/0:0+events23:50:130
396537299292,26cyclictest4086563-21kworker/0:0+events23:21:410
396537299292,26cyclictest4086563-21kworker/0:0+events22:17:450
396537299292,26cyclictest4051571-21kworker/0:2+events20:56:410
396537299292,23cyclictest4086563-21kworker/0:0+events23:55:110
396537299292,22cyclictest3970448-21kworker/0:0+events20:25:110
396537299291,27cyclictest4086563-21kworker/0:0+events23:33:160
396537299291,23cyclictest4086563-21kworker/0:0+events23:25:030
396537299291,22cyclictest4086563-21kworker/0:0+events21:25:150
396537299291,22cyclictest4086563-21kworker/0:0+events21:25:140
396537299290,28cyclictest4086563-21kworker/0:0+events22:00:370
3965379992823,3cyclictest4047491-21fschecks_time20:30:362
3965379992818,8cyclictest1704890-21systemd-journal19:55:502
3965376992821,5cyclictest4101408-21kworker/u8:3+flush-8:023:25:381
396537299289,18cyclictest3970448-21kworker/0:0+events20:00:460
396537299289,18cyclictest3970448-21kworker/0:0+events19:31:190
396537299287,20cyclictest3970448-21kworker/0:0+events19:24:450
396537299287,16cyclictest4086563-21kworker/0:0+events00:20:210
396537299286,17cyclictest4086563-21kworker/0:0+events22:50:150
396537299285,19cyclictest4086563-21kworker/0:0+events00:00:040
396537299282,21cyclictest4086563-21kworker/0:0+events22:20:090
396537299280,23cyclictest4086563-21kworker/0:0+events23:45:180
3965379992723,3cyclictest4006813-21kworker/u8:2+events_unbound20:54:272
3965379992723,3cyclictest3933392-21kworker/u8:1+events_unbound19:43:022
3965379992723,3cyclictest3933392-21kworker/u8:1+events_unbound19:43:012
3965379992723,2cyclictest4126726-21cut21:50:382
3965379992723,2cyclictest4042533-21cut20:25:352
3965379992722,4cyclictest4036777-21sendmail20:20:012
3965379992721,4cyclictest4105964-21unixbench_singl21:25:542
3965379992721,4cyclictest4105964-21unixbench_singl21:25:532
3965379992720,5cyclictest1704890-21systemd-journal19:40:002
3965376992721,4cyclictest4069173-21grep20:50:411
3965376992719,6cyclictest1704890-21systemd-journal22:55:001
396537299278,18cyclictest3970448-21kworker/0:0+events20:22:310
396537299277,19cyclictest4086563-21kworker/0:0+events00:29:350
396537299271,25cyclictest4086563-21kworker/0:0+events21:37:100
3965379992623,2cyclictest4011752-21sh19:55:002
3965379992622,3cyclictest3960308-21kworker/u8:3+events_unbound19:25:522
3965379992621,4cyclictest4006813-21kworker/u8:2+events_unbound21:40:532
3965379992621,3cyclictest4006813-21kworker/u8:2+events_unbound20:55:012
3965379992621,3cyclictest17025-21fschecks_count23:15:362
3965376992619,5cyclictest4119222-21kworker/u8:1+flush-8:022:10:481
3965379992521,2cyclictest52217-21fschecks_count23:50:352
3965379992521,2cyclictest4027454-21fschecks_time20:10:362
3965376992520,2cyclictest0-21swapper/123:55:001
3965376992520,2cyclictest0-21swapper/119:35:341
3965376992518,5cyclictest4119222-21kworker/u8:1+events_unbound22:55:361
39645442255,14sleep00-21swapper/019:05:500
3965379992416,6cyclictest1704890-21systemd-journal21:10:002
3965376992420,2cyclictest0-21swapper/100:10:441
3965376992419,3cyclictest401-21dbus-daemon22:40:001
3965376992417,4cyclictest4086787-21kworker/u8:0+events_unbound21:20:411
3965376992413,7cyclictest3986385-21sshd19:25:501
3965379992319,2cyclictest4020983-21needreboot20:00:442
396537999222,5cyclictest0-21swapper/222:28:582
396537999222,14cyclictest4146451-21apt-get22:10:342
3965379992218,2cyclictest4111793-21cpuspeed_turbos21:35:352
3965379992218,2cyclictest0-21swapper/221:55:502
396537999220,10cyclictest371rcuc/221:00:422
3965376992214,5cyclictest0-21swapper/123:45:511
3965376992214,5cyclictest0-21swapper/122:20:501
396537999216,2cyclictest161rcu_preempt19:10:532
396537999212,6cyclictest0-21swapper/220:40:222
3965379992117,2cyclictest21831-21aten2_r5power_p23:20:352
3965379992117,2cyclictest0-21swapper/223:40:002
3965379992113,6cyclictest31423-21grep23:30:012
3965379992111,2cyclictest161rcu_preempt23:55:352
396537999210,11cyclictest10149-21tr23:05:412
396537999210,11cyclictest10149-21tr23:05:402
3965376992117,3cyclictest0-21swapper/119:40:411
3965376992117,3cyclictest0-21swapper/119:40:411
3965376992116,4cyclictest4143669-21kthreadcore22:05:411
396537999203,7cyclictest0-21swapper/200:05:452
396537999203,2cyclictest161rcu_preempt21:20:522
3965379992017,2cyclictest3965367-21cyclictest00:01:242
3965379992015,3cyclictest90852-21unixbench_singl00:25:492
396537999200,5cyclictest4185275-21grep22:45:532
396537999200,11cyclictest94460-21kthreadcore00:30:432
3965376992013,4cyclictest0-21swapper/121:00:371
40935522190,5chrt0-21swapper/221:15:402
396537999194,2cyclictest421rcuog/222:40:462
396537999193,3cyclictest371rcuc/220:05:412
396537999192,2cyclictest0-21swapper/220:35:452
396537999192,12cyclictest3987350-21apt-get19:30:342
3965379991915,2cyclictest0-21swapper/219:20:462
396537999191,4cyclictest0-21swapper/222:05:422
3965379991910,7cyclictest4107836-21tr21:30:412
396537999190,10cyclictest371rcuc/200:20:412
39648452194,9sleep20-21swapper/219:05:512
396537999183,2cyclictest421rcuog/220:20:452
3965379991814,2cyclictest0-21swapper/222:00:492
3965379991814,2cyclictest0-21swapper/200:15:342
39605782182,9sleep30-21swapper/319:05:333
256182180,3sleep375-21kworker/3:1-events23:20:493
396537999176,2cyclictest161rcu_preempt00:10:392
3965379991713,2cyclictest0-21swapper/222:55:372
3965379991713,2cyclictest0-21swapper/219:45:342
396537999170,2cyclictest161rcu_preempt22:35:362
396537999160,2cyclictest161rcu_preempt22:15:382
990722130,6sleep30-21swapper/300:35:423
912542132,6sleep30-21swapper/300:30:013
566292132,6sleep30-21swapper/323:55:013
41759672132,6sleep30-21swapper/322:40:023
41586492133,6sleep30-21swapper/322:20:433
41553492132,7sleep30-21swapper/322:15:503
41261692132,6sleep30-21swapper/321:50:023
40806422132,7sleep30-21swapper/321:00:453
40308892132,7sleep30-21swapper/320:10:453
868912122,6sleep30-21swapper/300:25:353
816162122,6sleep30-21swapper/300:20:343
812242122,6sleep30-21swapper/300:19:103
506132122,6sleep30-21swapper/323:45:433
41856932122,5sleep30-21swapper/322:50:013
41815292122,6sleep30-21swapper/322:45:403
41751482122,7sleep30-21swapper/322:35:523
41655732122,6sleep30-21swapper/322:25:533
41469562123,5sleep30-21swapper/322:10:413
41258052122,6sleep30-21swapper/321:45:523
41106412122,6sleep30-21swapper/321:30:513
41016542122,6sleep30-21swapper/321:25:343
41016542122,6sleep30-21swapper/321:25:343
40967152122,6sleep30-21swapper/321:20:323
40865212122,5sleep30-21swapper/321:10:013
40759762122,6sleep30-21swapper/320:55:523
40422492122,6sleep30-21swapper/320:25:343
40410292122,6sleep30-21swapper/320:20:453
40261562122,6sleep30-21swapper/320:05:523
40176942122,6sleep30-21swapper/320:00:393
40094952122,6sleep30-21swapper/319:50:433
40008912122,6sleep30-21swapper/319:40:443
40008912122,6sleep30-21swapper/319:40:433
39960292123,6sleep30-21swapper/319:35:473
39771962122,5sleep30-21swapper/319:20:023
39658262122,6sleep30-21swapper/319:10:383
370412122,6sleep30-21swapper/323:35:393
21592122,6sleep30-21swapper/323:00:373
722162112,5sleep30-21swapper/300:10:383
668152111,6sleep30-21swapper/300:05:333
41909292112,5sleep30-21swapper/322:55:353
417892112,5sleep30-21swapper/323:40:333
41687482112,5sleep30-21swapper/322:30:433
41414542112,5sleep30-21swapper/322:05:033
41402702112,5sleep30-21swapper/322:00:453
41316862112,5sleep30-21swapper/321:55:363
41164312112,5sleep30-21swapper/321:40:033
41118252112,5sleep30-21swapper/321:35:373
40879162112,5sleep30-21swapper/321:10:413
40669682113,5sleep30-21swapper/320:50:233
40570982112,5sleep30-21swapper/320:40:363
40518542111,5sleep30-21swapper/320:35:023
40471222112,6sleep30-21swapper/320:30:083
40325382112,5sleep30-21swapper/320:15:393
40026402112,6sleep30-21swapper/319:45:383
39875922112,5sleep30-21swapper/319:30:373
39869612112,6sleep30-21swapper/319:28:513
39724522112,5sleep30-21swapper/319:15:323
313762112,5sleep30-21swapper/323:30:023
267532112,6sleep30-21swapper/323:25:373
211542112,5sleep30-21swapper/323:16:343
117532112,5sleep30-21swapper/323:10:323
617852102,5sleep30-21swapper/300:00:163
40629342100,5sleep30-21swapper/320:45:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional