You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 05:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Wed Aug 26, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28123212910,2sleep3575-21kworker/3:2-mm_percpu_wq00:17:423
25566862830,6chrt0-21swapper/220:02:382
2504153995547,6cyclictest2620289-21latency_hist21:05:001
2504153995547,6cyclictest2620289-21latency_hist21:05:001
2504151995445,6cyclictest2600531-21latency_hist20:45:010
26732202500,2chrt291ktimers/121:57:431
2504151994944,4cyclictest1436633-21kworker/u8:2+flush-8:020:22:510
2504153994839,7cyclictest2773833-21apt-get23:40:011
2504153994742,4cyclictest2649937-21kworker/u8:2+flush-8:022:57:461
2504153994742,3cyclictest2665020-21kworker/u8:1+flush-8:023:57:381
2504153994742,3cyclictest2509090-21kworker/u8:1+flush-8:020:02:501
2504153994741,3cyclictest1436633-21kworker/u8:2+events_unbound20:37:411
2504157994640,4cyclictest2665020-21kworker/u8:1+events_unbound22:00:002
2504153994640,4cyclictest2649937-21kworker/u8:2+events_unbound22:22:401
2504153994540,3cyclictest2595834-21kworker/u8:0+events_unbound20:42:381
2504151994540,4cyclictest2509090-21kworker/u8:1+flush-8:019:22:400
2504157994440,3cyclictest2803711-21kworker/u8:1+events_unbound00:27:402
2504153994435,7cyclictest2704421-21cp22:30:001
2504151994436,6cyclictest2769317-21/usr/sbin/munin23:37:460
2504151994435,7cyclictest275-21systemd-journal21:03:200
2504151994435,7cyclictest275-21systemd-journal21:03:200
2504153994337,4cyclictest2536416-21/usr/sbin/munin19:42:521
2504153994337,4cyclictest1696005-21kworker/u8:3+flush-8:019:12:381
2504151994338,4cyclictest2585991-21kworker/u8:3+events_unbound20:47:550
24994962433,35sleep10-21swapper/119:07:001
2504153994237,4cyclictest2541241-21kworker/u8:0+events_unbound20:27:471
2504153994237,4cyclictest1436633-21kworker/u8:2+flush-8:019:32:471
2504153994236,5cyclictest2649937-21kworker/u8:2+flush-8:023:47:371
2504153994236,5cyclictest2649937-21kworker/u8:2+flush-8:022:47:461
2504153994235,4cyclictest2818960-21latency_hist00:25:011
2504153994233,7cyclictest275-21systemd-journal21:32:191
2504153994233,7cyclictest2541396-21/usr/sbin/munin19:47:451
2504157994136,4cyclictest2595834-21kworker/u8:0+flush-8:021:30:212
2504153994136,4cyclictest2595834-21kworker/u8:0+events_unbound21:27:381
2504153994136,4cyclictest2541241-21kworker/u8:0+events_unbound20:17:521
2504153994134,5cyclictest2649937-21kworker/u8:2+flush-8:023:12:491
2504157994035,4cyclictest2509090-21kworker/u8:1+events_unbound19:27:492
2504153994035,4cyclictest2541241-21kworker/u8:0+flush-8:019:52:501
2504153994033,5cyclictest2764405-21apt-get23:32:321
2504153994031,6cyclictest2570603-21latency_hist20:15:011
2504151994031,6cyclictest2644988-21latency_hist21:30:010
2504153993935,3cyclictest2665020-21kworker/u8:1+events_unbound22:52:501
2504153993934,4cyclictest2585991-21kworker/u8:3+events_unbound21:22:551
2504153993933,5cyclictest2665020-21kworker/u8:1+flush-8:022:32:491
2504153993933,5cyclictest2509090-21kworker/u8:1+flush-8:019:22:511
2504151993930,6cyclictest2535980-21latency_hist19:40:010
2504157993833,4cyclictest2585991-21kworker/u8:3+events_unbound22:10:002
2504153993834,3cyclictest2665020-21kworker/u8:1+flush-8:023:42:461
2504153993833,4cyclictest2665020-21kworker/u8:1+events_unbound23:27:471
2504153993832,4cyclictest2585991-21kworker/u8:3+flush-8:021:42:411
2504153993830,6cyclictest2824329-21/usr/sbin/munin00:32:511
2504151993831,5cyclictest2610425-21latency_hist20:55:010
2504151993830,6cyclictest2768904-21latency_hist23:35:010
2504157993733,3cyclictest2829108-21kworker/u8:3+events_unbound00:37:452
2504157993733,3cyclictest1436633-21kworker/u8:2+events_unbound19:15:052
2504157993731,4cyclictest275-21systemd-journal19:48:092
2504157993730,5cyclictest2828847-21turbostat.cron00:35:002
2504153993731,4cyclictest2704785-21kworker/u8:0+events_unbound23:07:451
2504151993728,7cyclictest2804537-21/usr/sbin/munin00:12:520
2504157993628,6cyclictest2749183-21turbostat.cron23:15:002
2504153993626,8cyclictest2564902-21ntpq20:07:451
2504157993526,7cyclictest2675385-21diskstats22:02:352
2504153993530,4cyclictest2509090-21kworker/u8:1+events_unbound19:37:511
2504153993529,5cyclictest1436633-21kworker/u8:2+flush-8:021:17:411
2504153993529,4cyclictest275-21systemd-journal00:14:231
2504153993529,4cyclictest2665020-21kworker/u8:1+flush-8:023:52:431
2504153993528,5cyclictest2814004-21latency_hist00:20:011
2504153993528,5cyclictest2472778-21kworker/u8:0+flush-8:019:17:381
2504151993523,9cyclictest275-21systemd-journal22:20:000
2504153993430,2cyclictest2694447-21systemctl22:20:001
2504153993429,3cyclictest2585991-21kworker/u8:3+flush-8:020:57:461
2504151993428,4cyclictest1696005-21kworker/u8:3+flush-8:019:12:500
2504157993326,5cyclictest2590617-21cat20:35:002
2504153993328,4cyclictest2585991-21kworker/u8:3+events_unbound22:12:501
2504153993325,6cyclictest275-21systemd-journal22:05:201
2504153993325,6cyclictest2704785-21kworker/u8:0+events_unbound23:22:191
2504153993325,6cyclictest2704785-21kworker/u8:0+events_unbound23:22:191
2504153993324,7cyclictest2555038-21ntpq19:57:451
2504153993323,8cyclictest275-21systemd-journal20:55:001
2504151993326,5cyclictest275-21systemd-journal22:47:210
2504151993323,8cyclictest275-21systemd-journal23:30:000
2504153993226,4cyclictest2794130-21kworker/u8:4+flush-8:000:07:441
2504153993225,6cyclictest2665020-21kworker/u8:1+flush-8:021:52:471
2504151993223,7cyclictest275-21systemd-journal21:55:000
2504151993223,6cyclictest275-21systemd-journal23:05:000
2504151993222,8cyclictest2744941-21diskstats23:12:350
2504157993126,4cyclictest2665020-21kworker/u8:1+events_unbound23:35:182
2504157993125,4cyclictest2649937-21kworker/u8:2+events_unbound22:45:022
2504153993126,3cyclictest2718665-21ntpq22:42:461
2504151993128,2cyclictest2729244-21systemd-run22:55:000
2504151993126,3cyclictest2585991-21kworker/u8:3+events_unbound21:37:500
2504151993126,3cyclictest2525902-21kworker/u8:3+flush-8:019:42:410
2504151993124,5cyclictest2809508-21apt-get00:17:320
2504151993121,8cyclictest2704977-21munin-node22:32:340
26287862300,2sleep00-21swapper/021:12:510
2504157993022,6cyclictest2585634-21cat20:30:012
2504153993026,3cyclictest2585991-21kworker/u8:3+flush-8:021:13:011
2504153993024,4cyclictest2585991-21kworker/u8:3+flush-8:021:37:561
2504153993023,5cyclictest275-21systemd-journal20:47:191
2504151993027,2cyclictest2585991-21kworker/u8:3+events_unbound21:22:530
2504151993026,3cyclictest2509090-21kworker/u8:1+flush-8:019:34:240
2504151993025,4cyclictest2509090-21kworker/u8:1+flush-8:020:02:370
2504151993022,6cyclictest275-21systemd-journal19:55:000
2504157992917,9cyclictest2615326-21awk21:00:002
2504153992922,5cyclictest2665020-21kworker/u8:1+flush-8:023:02:521
2504153992913,10cyclictest275-21systemd-journal20:30:031
2504151992919,8cyclictest2793872-40logrotate00:00:010
2504157992824,3cyclictest2509090-21kworker/u8:1+events_unbound19:24:282
2504157992824,2cyclictest2635591-21aten2_r5power_e21:22:362
2504157992823,3cyclictest2659836-21idleruntime-cro21:45:002
2504157992822,4cyclictest2753291-21needreboot23:17:432
2504153992824,3cyclictest2803711-21kworker/u8:1+events_unbound00:27:501
2504153992822,4cyclictest2829108-21kworker/u8:3+flush-8:000:37:441
2504153992821,5cyclictest1436633-21kworker/u8:2+flush-8:020:22:481
2504151992822,5cyclictest2704785-21kworker/u8:0+flush-8:023:22:450
2504151992822,5cyclictest2704785-21kworker/u8:0+flush-8:023:22:450
2504151992822,4cyclictest2472778-21kworker/u8:0+flush-8:019:17:500
2504151992821,5cyclictest275-21systemd-journal23:00:000
2504157992724,2cyclictest2620011-21unixbench_multi21:02:562
2504157992724,2cyclictest2620011-21unixbench_multi21:02:562
2504157992722,3cyclictest2774659-21aten2_r5power_p23:42:342
2504157992720,5cyclictest2784497-21grep23:52:342
2504151992723,3cyclictest2818943-21cron00:25:000
2504151992723,2cyclictest2561525-21fschecks_time20:07:370
2504151992721,4cyclictest2829108-21kworker/u8:3+events_unbound00:37:530
2504151992716,9cyclictest2798928-21apt-get00:05:000
250415199270,26cyclictest2664806-21kworker/0:2+events22:22:520
2504157992622,3cyclictest2649937-21kworker/u8:2+flush-8:023:05:112
2504157992622,2cyclictest2779763-21cpuspeed_turbos23:47:352
2504157992618,6cyclictest2803312-21needreboot00:07:452
2504153992621,4cyclictest2665020-21kworker/u8:1+flush-8:022:00:311
2504153992621,4cyclictest2525855-21unixbench_singl19:27:511
2504153992621,3cyclictest2662442-21kthreadcore21:47:421
2504153992618,6cyclictest2649937-21kworker/u8:2+flush-8:023:17:461
2504151992622,3cyclictest2799832-21cpuspeed_turbos00:07:360
2504151992622,2cyclictest0-21swapper/020:15:000
2504151992621,4cyclictest2665020-21kworker/u8:1+events_unbound22:07:390
2504151992621,4cyclictest2595834-21kworker/u8:0+flush-8:021:07:370
2504151992621,4cyclictest2509090-21kworker/u8:1+events_unbound19:57:500
2504151992621,3cyclictest2591783-21ps20:37:410
2504151992620,5cyclictest2649937-21kworker/u8:2+flush-8:022:42:500
2504151992620,5cyclictest2590431-21unixbench_singl20:33:060
2504151992620,4cyclictest2803711-21kworker/u8:1+events_unbound00:30:000
2504151992617,7cyclictest2749603-21/usr/sbin/munin23:17:440
250415799252,4cyclictest0-21swapper/223:22:352
250415799252,4cyclictest0-21swapper/223:22:352
2504157992522,2cyclictest20823-21cron22:20:002
2504157992521,3cyclictest2699937-21aten2_r5power_p22:27:322
2504157992521,3cyclictest2640287-21kworker/u8:1+flush-8:021:28:172
2504157992520,3cyclictest2504535-21aten2_r5power_c19:12:352
2504153992520,3cyclictest2799128-21apt-key00:05:011
2504153992514,8cyclictest275-21systemd-journal22:35:191
2504151992521,2cyclictest0-21swapper/020:30:010
2504151992520,4cyclictest2640287-21kworker/u8:1+flush-8:021:32:510
2504151992519,5cyclictest2665020-21kworker/u8:1+flush-8:023:52:470
2504151992519,4cyclictest2699881-21aten2_r5power_c22:27:320
2504151992519,4cyclictest2585991-21kworker/u8:3+flush-8:020:57:500
2504151992519,4cyclictest0-21swapper/022:40:010
2504151992518,6cyclictest2803711-21kworker/u8:1+flush-8:000:32:520
2504151992514,3cyclictest0-21swapper/023:40:020
2504157992421,2cyclictest2759717-21cpuspeed_turbos23:27:352
2504157992421,2cyclictest2531839-21fschecks_time19:37:402
250415199242,21cyclictest0-21swapper/019:47:520
2504151992420,3cyclictest2674399-21unixbench_multi21:57:520
2504151992417,5cyclictest0-21swapper/023:47:350
2504157992319,3cyclictest2704785-21kworker/u8:0+flush-8:023:33:002
2504157992318,3cyclictest2709299-21unixbench_singl22:32:552
2504153992316,5cyclictest2595834-21kworker/u8:0+events_unbound21:07:401
2504151992319,3cyclictest1436633-21kworker/u8:2+flush-8:020:17:560
2504151992315,6cyclictest2739868-21aten2_r5power_p23:07:340
24993662232,5sleep00-21swapper/019:05:180
250415799229,9cyclictest2724730-21apt-get22:52:332
2504157992219,2cyclictest2793884-21systemctl00:00:002
2504157992218,3cyclictest2541241-21kworker/u8:0+events_unbound20:18:202
2504157992212,8cyclictest2797968-21grep00:02:442
2504157992212,8cyclictest2531059-21awk19:35:002
250415799221,11cyclictest2609925-21sensors_fan20:52:532
2504151992218,2cyclictest0-21swapper/021:45:000
2504151992217,3cyclictest2634808-21unixbench_multi21:17:540
2504151992217,3cyclictest0-21swapper/019:27:340
250415799217,2cyclictest25701681switchtime20:12:522
2504157992117,3cyclictest2734164-21latency_hist23:00:002
2504157992117,3cyclictest2600260-21unixbench_multi20:42:532
2504157992117,2cyclictest2629919-21unixbench_singl21:13:012
2504151992116,3cyclictest0-21swapper/021:47:370
27953002200,3sleep3575-21kworker/3:2+events00:02:413
250415799202,13cyclictest2694898-21apt-get22:22:352
2504157992016,2cyclictest2735069-21cpuspeed_turbos23:02:352
2504157992013,2cyclictest161rcu_preempt20:37:442
250415799201,2cyclictest161rcu_preempt19:42:552
2504157992010,8cyclictest2577976-21tr20:22:432
2504151992016,2cyclictest0-21swapper/022:12:540
2504151992015,3cyclictest2675168-21aten2_r5power_c22:02:330
24994922204,11sleep20-21swapper/219:06:572
2504157991914,3cyclictest0-21swapper/221:52:532
2504157991911,6cyclictest2549483-21tr19:52:432
2504157991910,7cyclictest2654883-21cron21:40:002
2504157991910,7cyclictest2604433-21tr20:47:462
24995182193,9sleep30-21swapper/319:07:183
250415799182,2cyclictest421rcuog/200:12:452
2504157991814,2cyclictest0-21swapper/221:09:402
250415799181,2cyclictest0-21swapper/219:57:342
2504157991811,5cyclictest0-21swapper/220:07:402
250415799179,6cyclictest2689498-21sh22:15:002
2504157991712,3cyclictest2711549-21kthreadcore22:37:422
2504157991712,3cyclictest0-21swapper/200:17:522
250415799162,5cyclictest0-21swapper/221:15:522
250415799162,2cyclictest0-21swapper/222:40:512
2504157991611,3cyclictest0-21swapper/200:22:432
27109342156,5sleep30-21swapper/322:37:433
250415799156,6cyclictest2660454-21cpuspeed21:47:352
25614842145,5sleep30-21swapper/320:07:383
28331012132,7sleep30-21swapper/300:37:493
27898532132,7sleep30-21swapper/323:57:403
27840262132,6sleep30-21swapper/323:50:023
26845652132,6sleep30-21swapper/322:10:023
26294052132,7sleep30-21swapper/321:12:553
26244212132,7sleep30-21swapper/321:07:453
26195932132,7sleep30-21swapper/321:02:523
26195932132,7sleep30-21swapper/321:02:523
25757382132,6sleep30-21swapper/320:20:023
25607182130,7sleep30-21swapper/320:05:013
25508942132,6sleep30-21swapper/319:55:023
25325312132,7sleep30-21swapper/319:37:443
28189692122,5sleep30-21swapper/300:25:013
27804232122,6sleep30-21swapper/323:47:413
27789902122,6sleep30-21swapper/323:43:563
27693162122,6sleep30-21swapper/323:37:343
27646472122,6sleep30-21swapper/323:32:363
27598842124,5sleep30-21swapper/323:27:393
27540842123,5sleep30-21swapper/323:20:003
27301872122,5sleep30-21swapper/322:57:413
27193932122,5sleep30-21swapper/322:45:013
27075622122,6sleep30-21swapper/322:32:453
27001032122,6sleep30-21swapper/322:27:353
26957802122,6sleep30-21swapper/322:22:423
26899482122,6sleep30-21swapper/322:17:363
26739012122,6sleep30-21swapper/321:57:463
26598442122,5sleep30-21swapper/321:45:013
26552172123,6sleep30-21swapper/321:41:203
26503652122,6sleep30-21swapper/321:37:353
26391222122,6sleep30-21swapper/321:22:463
26111212122,6sleep30-21swapper/320:57:423
26062852122,6sleep30-21swapper/320:52:403
26001152122,6sleep30-21swapper/320:42:533
25949792122,6sleep30-21swapper/320:37:523
25865612122,6sleep30-21swapper/320:32:543
25850092122,6sleep30-21swapper/320:27:523
25705842122,6sleep30-21swapper/320:15:003
25364262124,5sleep30-21swapper/319:42:343
25260842122,6sleep30-21swapper/319:30:023
25229562122,6sleep30-21swapper/319:27:413
25163842122,6sleep30-21swapper/319:22:353
25043832122,6sleep30-21swapper/319:11:203
28241602112,5sleep30-21swapper/300:30:233
28144962112,5sleep30-21swapper/300:22:373
28085762112,5sleep30-21swapper/300:12:533
27994812112,5sleep30-21swapper/300:06:193
27543642112,5sleep30-21swapper/323:20:203
27543642112,5sleep30-21swapper/323:20:203
27442762112,5sleep30-21swapper/323:10:013
27396852112,6sleep30-21swapper/323:06:363
27347902112,5sleep30-21swapper/323:02:063
27287412111,6sleep30-21swapper/322:52:513
27148132112,6sleep30-21swapper/322:41:563
26826372112,5sleep30-21swapper/322:07:453
26749192112,5sleep30-21swapper/322:00:023
26651492112,6sleep30-21swapper/321:51:523
26463072112,6sleep30-21swapper/321:32:413
26442042112,6sleep30-21swapper/321:27:483
26350562111,5sleep30-21swapper/321:20:013
26007522112,5sleep30-21swapper/320:45:023
25710192112,5sleep30-21swapper/320:17:333
25499632112,5sleep30-21swapper/319:52:453
25412012111,5sleep30-21swapper/319:45:023
25097542112,5sleep30-21swapper/319:15:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional