You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 05:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Aug 08, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
335394521790,2sleep30-21swapper/320:32:563
33031542540,5sleep13303152-21fschecks_time19:42:501
3270563995242,7cyclictest3376701-21latency_hist20:55:011
3270563995143,5cyclictest3555460-21apt-get23:57:451
3270556995146,3cyclictest3450978-21kworker/u8:2+events_unbound22:57:470
3270556995144,5cyclictest3450978-21kworker/u8:2+events_unbound22:47:460
3270556995045,4cyclictest3411612-21kworker/u8:1+events_unbound23:22:500
3270556994945,3cyclictest3260607-21kworker/u8:2+events_unbound19:18:040
33303952480,3sleep1281rcuc/120:07:541
3270556994843,4cyclictest3431197-21kworker/u8:0+flush-8:023:13:040
34893092470,2sleep130-21ksoftirqd/122:47:551
3270556994739,6cyclictest3336963-21latency_hist20:15:010
3270556994640,4cyclictest3260607-21kworker/u8:2+events_unbound21:30:010
35267842440,3sleep3575-21kworker/3:2+events23:27:533
35267842440,3sleep3575-21kworker/3:2+events23:27:533
3270568994433,8cyclictest3460837-21cat22:19:592
3270568994428,15cyclictest3362058-21base6420:40:012
3270556994439,4cyclictest3270408-21kworker/u8:1+events_unbound19:30:000
3270556994435,7cyclictest3311560-21ntpq19:47:550
3270556994337,4cyclictest3431197-21kworker/u8:0+events_unbound23:05:010
3270563994231,8cyclictest275-21systemd-journal21:25:021
3270556994236,4cyclictest3411612-21kworker/u8:1+events_unbound23:27:490
3270556994236,4cyclictest3411612-21kworker/u8:1+events_unbound23:27:490
327034924226,10sleep10-21swapper/119:09:451
3270556994137,3cyclictest3431197-21kworker/u8:0+events_unbound22:52:450
3270556994034,4cyclictest3411612-21kworker/u8:1+events_unbound23:48:000
3270556994033,5cyclictest3560276-21kworker/u8:4+events_unbound00:02:470
3270568993920,16cyclictest3341878-21awk20:20:002
3270556993935,3cyclictest3560276-21kworker/u8:4+events_unbound00:17:470
3270556993935,3cyclictest3302610-21kworker/u8:3+events_unbound19:58:040
3270556993934,3cyclictest3260607-21kworker/u8:2+events_unbound19:23:070
3270556993933,4cyclictest3550301-21kworker/u8:0+events_unbound23:52:520
3270556993933,4cyclictest3356530-21kworker/u8:1+events_unbound21:07:500
3270556993933,4cyclictest275-21systemd-journal21:34:280
3270556993834,3cyclictest3302610-21kworker/u8:3+flush-8:022:07:540
3270556993833,3cyclictest3302610-21kworker/u8:3+events_unbound21:58:050
3270556993832,4cyclictest3534642-21kworker/u8:2+events_unbound23:38:030
3270556993832,4cyclictest3450978-21kworker/u8:2+events_unbound22:12:490
3270556993832,4cyclictest3450978-21kworker/u8:2+events_unbound22:12:490
3270556993832,4cyclictest3260607-21kworker/u8:2+events_unbound21:47:500
3270556993832,4cyclictest3260607-21kworker/u8:2+events_unbound20:18:000
3270568993724,11cyclictest3283371-21fschecks_time19:22:512
3270563993728,6cyclictest275-21systemd-journal19:24:401
3270556993731,4cyclictest3450978-21kworker/u8:2+events_unbound22:23:000
3270568993622,12cyclictest3441901-21fschecks_time22:02:502
3270568993621,13cyclictest0-21swapper/221:20:002
327056899361,28cyclictest3317689-21apt-get19:57:482
3270563993628,6cyclictest3550503-21/usr/sbin/munin23:53:001
3270563993626,8cyclictest3495650-21latency_hist22:55:021
3270563993625,9cyclictest3499827-21ntpq22:57:581
3270556993632,3cyclictest3411612-21kworker/u8:1+flush-8:021:43:070
3270556993631,3cyclictest3411612-21kworker/u8:1+events_unbound00:28:060
3270556993630,4cyclictest3260607-21kworker/u8:2+events_unbound20:43:050
3270568993523,10cyclictest3322906-21fschecks_count20:02:502
327056899352,25cyclictest3387114-21apt-get21:07:482
3270568993517,15cyclictest3336972-21awk20:15:002
327056899351,27cyclictest3481275-21apt-get22:42:462
327056899350,19cyclictest3509526-21/usr/sbin/munin23:07:562
3270563993528,5cyclictest275-21systemd-journal19:27:401
3270556993532,2cyclictest3302610-21kworker/u8:3+events_unbound21:53:030
3270556993531,3cyclictest3302610-21kworker/u8:3+events_unbound20:28:040
3270556993530,4cyclictest3302610-21kworker/u8:3+flush-8:019:53:050
3270556993530,4cyclictest3297292-21kworker/u8:0+events_unbound20:02:570
3270556993529,4cyclictest3356530-21kworker/u8:1+events_unbound20:38:040
3270556993529,4cyclictest3302610-21kworker/u8:3+events_unbound21:15:000
3270556993529,4cyclictest3302610-21kworker/u8:3+events_unbound21:15:000
35886152340,6sleep3451irq_work/300:27:563
35213402340,5sleep3451irq_work/323:22:523
3270568993421,11cyclictest3421102-21grep21:38:052
3270563993424,8cyclictest275-21systemd-journal22:05:001
3270556993430,3cyclictest3569967-21kworker/u8:0+events_unbound00:12:480
3270556993430,3cyclictest3515616-21kworker/u8:3+events_unbound23:17:470
3270556993430,3cyclictest3431197-21kworker/u8:0+flush-8:022:18:040
3270556993430,3cyclictest3302610-21kworker/u8:3+flush-8:021:23:050
3270556993429,4cyclictest3569967-21kworker/u8:0+events_unbound00:31:180
3270556993429,4cyclictest3569967-21kworker/u8:0+events_unbound00:31:180
3270556993429,3cyclictest3356530-21kworker/u8:1+events_unbound21:03:050
3270556993428,5cyclictest3534642-21kworker/u8:2+flush-8:023:42:510
3270556993427,2cyclictest3260607-21kworker/u8:2+events_unbound21:16:460
327056899333,23cyclictest3406877-21apt-get21:27:462
3270568993315,15cyclictest3312120-21unixbench_singl19:48:032
327056899331,24cyclictest3515842-21apt-get23:17:452
327056899331,24cyclictest3312778-21apt-get19:52:482
3270563993326,5cyclictest3426252-21latency_hist21:45:011
3270556993329,3cyclictest3450978-21kworker/u8:2+flush-8:022:42:510
327056899321,24cyclictest3525733-21apt-get23:27:462
327056899321,24cyclictest3525733-21apt-get23:27:462
3270556993228,3cyclictest3450978-21kworker/u8:2+flush-8:022:33:040
3270556993228,3cyclictest3260607-21kworker/u8:2+events_unbound20:23:070
3270556993225,5cyclictest275-21systemd-journal20:56:460
3270556993224,6cyclictest3356530-21kworker/u8:1+events_unbound20:48:000
34602382310,2sleep1161rcu_preempt22:18:021
3270568993126,3cyclictest3511258-21fschecks_time23:12:502
327056899312,22cyclictest3302842-21apt-get19:42:482
327056899311,23cyclictest3287967-21apt-get19:27:462
3270563993126,3cyclictest3381723-21idleruntime-cro21:00:001
3270556993127,3cyclictest3302610-21kworker/u8:3+flush-8:020:32:540
3270556993127,3cyclictest3302610-21kworker/u8:3+events_unbound21:38:470
3270556993127,3cyclictest3297292-21kworker/u8:0+flush-8:019:43:050
3270556993127,3cyclictest3270408-21kworker/u8:1+events_unbound19:33:030
3270556993126,4cyclictest3411612-21kworker/u8:1+events_unbound22:37:530
3270556993124,5cyclictest3431197-21kworker/u8:0+events_unbound22:27:480
32703182314,10sleep20-21swapper/219:09:222
327056899302,20cyclictest3292902-21apt-get19:32:472
3270568993015,13cyclictest0-21swapper/221:32:572
327056899301,24cyclictest3477031-21ps22:37:532
327056899301,22cyclictest3565500-21apt-get00:07:492
3270568993012,15cyclictest3406430-21awk21:25:002
3270563993025,3cyclictest3450978-21kworker/u8:2+flush-8:023:00:111
3270563993022,6cyclictest3436569-21/usr/sbin/munin21:58:031
3270556993026,3cyclictest3560276-21kworker/u8:4+flush-8:000:37:510
3270556993026,3cyclictest3560276-21kworker/u8:4+flush-8:000:37:510
3270556993026,3cyclictest3450978-21kworker/u8:2+flush-8:023:08:040
3270556993024,4cyclictest3411612-21kworker/u8:1+events_unbound00:07:590
3270556993023,5cyclictest3297292-21kworker/u8:0+events_unbound19:38:010
3270556993022,6cyclictest3233783-21kworker/u8:3+events_unbound19:12:500
327056899291,24cyclictest3384334-21ps21:02:562
327056899291,23cyclictest3363266-21ps20:42:552
3270563992922,6cyclictest3475643-21cron22:35:001
3270563992921,6cyclictest3584031-21ntpq00:22:581
3270563992921,6cyclictest3431641-21apt-get21:52:461
3270563992920,7cyclictest3599815-21cat00:40:001
3270563992920,7cyclictest3599815-21cat00:40:001
3270556992922,5cyclictest3302610-21kworker/u8:3+events_unbound20:52:470
327056899282,20cyclictest3277846-21apt-get19:17:472
3270568992821,6cyclictest3554998-21cron23:55:012
3270568992814,12cyclictest0-21swapper/223:48:032
3270568992814,12cyclictest0-21swapper/220:52:592
3270568992812,13cyclictest3436819-21cpuspeed_turbos21:57:482
3270568992810,16cyclictest3394079-21tr21:12:542
3270568992810,16cyclictest3394079-21tr21:12:542
327056899280,9cyclictest3331591-21/usr/sbin/munin20:08:032
327056899280,10cyclictest3431616-21/usr/sbin/munin21:52:462
3270563992824,3cyclictest3321374-21needreboot19:57:571
3270556992825,2cyclictest3411612-21kworker/u8:1+events_unbound00:23:040
327056899272,18cyclictest3342354-21apt-get20:22:472
327056899271,9cyclictest3451673-21/usr/sbin/munin22:12:502
327056899271,9cyclictest3451673-21/usr/sbin/munin22:12:502
3270568992714,11cyclictest0-21swapper/220:46:352
327056899271,23cyclictest3589061-21ntpq00:28:012
327056899271,23cyclictest3584174-21/usr/sbin/munin00:23:022
327056899271,22cyclictest3572577-21ps00:12:532
327056899271,22cyclictest3427206-21/usr/sbin/munin21:47:522
327056899271,20cyclictest3531224-21irqcore23:32:512
3270568992711,14cyclictest3523784-21tr23:22:552
327056899270,9cyclictest3575390-21/usr/sbin/munin00:17:472
327056899270,8cyclictest3474963-21ntpq22:32:592
3270563992723,3cyclictest3391592-21systemd-run21:10:001
3270563992723,3cyclictest3280010-21kthreadcore19:17:541
3270563992722,3cyclictest20823-21cron23:25:001
3270563992719,6cyclictest275-21systemd-journal23:35:451
3270563992717,8cyclictest3485029-21ntpq22:42:571
3270556992723,3cyclictest3550301-21kworker/u8:0+flush-8:023:55:010
3270556992723,3cyclictest3431197-21kworker/u8:0+events_unbound22:03:080
3270556992723,3cyclictest3260607-21kworker/u8:2+events_unbound20:08:060
327056899266,18cyclictest3351484-21unixbench_multi20:28:062
327056899264,20cyclictest3273600-21cut19:12:552
327056899261,8cyclictest3352571-21fw_packets20:32:512
3270568992613,11cyclictest0-21swapper/222:55:432
327056899261,21cyclictest3559299-21ntpq23:58:012
327056899261,21cyclictest3446729-21df_inode22:07:492
327056899261,20cyclictest3540339-21latency_hist23:40:012
3270568992610,13cyclictest0-21swapper/222:22:512
327056899260,9cyclictest3486582-21/usr/sbin/munin22:47:502
327056899260,9cyclictest3298124-21/usr/sbin/munin19:37:502
327056899260,8cyclictest3590358-21/usr/sbin/munin00:32:472
327056899260,8cyclictest3590358-21/usr/sbin/munin00:32:472
327056899260,8cyclictest3494899-21ntpq22:52:582
3270563992622,3cyclictest401-21dbus-daemon21:50:001
3270563992622,2cyclictest0-21swapper/119:32:531
3270563992621,3cyclictest3396294-21unixbench_multi21:13:061
3270563992621,3cyclictest3396294-21unixbench_multi21:13:051
3270563992619,5cyclictest3544351-21ntpq23:42:581
3270563992619,5cyclictest3371057-21ntp_kernel_pll_20:48:001
3270556992619,5cyclictest3530665-21/usr/sbin/munin23:32:560
327056899251,8cyclictest3564325-21ntpq00:02:582
3270568992511,12cyclictest3505507-21idleruntime-cro23:05:002
327056899250,8cyclictest3377626-21/usr/sbin/munin20:57:512
327056899250,23cyclictest3425365-21/usr/sbin/munin21:42:582
3270563992521,3cyclictest3526456-21irqrtprio23:27:531
3270563992521,3cyclictest3526456-21irqrtprio23:27:531
3270563992521,3cyclictest3326916-21unixbench_singl20:03:051
3270563992521,2cyclictest0-21swapper/122:12:531
3270563992521,2cyclictest0-21swapper/122:12:531
3270563992520,3cyclictest3476344-21cpuspeed_turbos22:37:481
3270563992520,3cyclictest3317222-21ls19:55:001
3270563992519,4cyclictest3470698-21latency_hist22:30:011
34857692240,4sleep3481ktimers/322:45:003
327056899241,21cyclictest3469933-21/usr/sbin/munin22:27:592
327056899241,19cyclictest3598876-21ntpq00:38:002
327056899241,19cyclictest3598876-21ntpq00:38:002
3270563992420,2cyclictest3270549-21cyclictest23:12:521
3270563992420,2cyclictest0-21swapper/122:21:451
3270563992419,3cyclictest0-21swapper/120:37:531
3270563992319,3cyclictest3565029-21cron00:05:001
3270563992319,3cyclictest3298254-21if_enp3s019:37:511
3270563992319,2cyclictest0-21swapper/122:07:501
3270563992319,2cyclictest0-21swapper/121:32:501
3270563992319,2cyclictest0-21swapper/120:33:041
3270563992319,2cyclictest0-21swapper/120:17:471
3270563992317,4cyclictest3569746-21grep00:08:071
3270563992315,5cyclictest3421298-21latency_hist21:40:021
327056899220,19cyclictest3540976-21grep23:42:502
3270563992218,3cyclictest3534987-21unixbench_singl23:33:041
3270563992218,2cyclictest0-21swapper/119:48:011
3270563992218,2cyclictest0-21swapper/100:20:001
3270563992217,3cyclictest0-21swapper/121:27:531
3270563992217,3cyclictest0-21swapper/121:02:541
3270563992216,4cyclictest3361669-21http20:40:011
3270563992215,5cyclictest0-21swapper/120:23:071
3270563992213,7cyclictest275-21systemd-journal20:12:451
3270563992211,8cyclictest3274769-21ntpq19:13:001
327056399218,9cyclictest3545569-21apt-get23:47:481
3270563992117,2cyclictest0-21swapper/123:07:521
3270563992117,2cyclictest0-21swapper/121:17:561
3270563992117,2cyclictest0-21swapper/100:28:051
3270563992117,2cyclictest0-21swapper/100:12:481
3270563992116,3cyclictest0-21swapper/120:27:511
3270563992016,2cyclictest0-21swapper/100:32:501
3270563992016,2cyclictest0-21swapper/100:32:491
3270563992013,5cyclictest275-21systemd-journal23:16:451
32697682183,10sleep00-21swapper/019:08:030
32658552183,8sleep30-21swapper/319:06:223
33127812172,9sleep30-21swapper/319:52:483
34343102160,2sleep3481ktimers/321:52:563
33870892152,8sleep30-21swapper/321:07:473
32780932155,6sleep30-21swapper/319:17:493
35953892132,7sleep30-21swapper/300:37:533
35953892132,7sleep30-21swapper/300:37:523
35403362132,6sleep30-21swapper/323:40:023
34949132132,7sleep30-21swapper/322:53:003
33571072132,7sleep30-21swapper/320:37:463
33324822132,7sleep30-21swapper/320:12:463
35900312122,6sleep30-21swapper/300:31:183
35900312122,6sleep30-21swapper/300:31:183
35749222122,5sleep30-21swapper/300:15:013
35559162122,6sleep30-21swapper/323:57:523
35500772122,5sleep30-21swapper/323:50:013
35356512122,6sleep30-21swapper/323:37:453
35306642122,6sleep30-21swapper/323:32:443
35198692122,6sleep30-21swapper/323:18:033
35104622122,5sleep30-21swapper/323:10:023
35007792122,6sleep30-21swapper/323:00:023
34754792122,6sleep30-21swapper/322:33:063
34517632122,6sleep30-21swapper/322:12:513
34517632122,6sleep30-21swapper/322:12:513
34470002122,6sleep30-21swapper/322:07:543
34364152122,6sleep30-21swapper/321:55:463
34213272122,5sleep30-21swapper/321:40:013
34208232123,6sleep30-21swapper/321:38:053
34115792122,6sleep30-21swapper/321:30:023
33821512122,6sleep30-21swapper/321:02:473
33673912123,6sleep30-21swapper/320:47:493
33626012122,6sleep30-21swapper/320:42:523
33511022122,6sleep30-21swapper/320:28:033
33075852122,6sleep30-21swapper/319:45:023
32978432122,6sleep30-21swapper/319:37:483
32931462122,6sleep30-21swapper/319:32:503
32709122122,6sleep30-21swapper/319:12:483
35847342112,6sleep30-21swapper/300:24:403
35702102112,6sleep30-21swapper/300:10:323
35690642112,5sleep30-21swapper/300:07:593
35604672112,5sleep30-21swapper/300:02:453
35453842112,6sleep30-21swapper/323:45:443
35060432112,5sleep30-21swapper/323:07:483
34960762112,5sleep30-21swapper/322:57:463
34857692112,5sleep30-21swapper/322:45:013
34758762112,5sleep30-21swapper/322:35:023
34704822112,6sleep30-21swapper/322:28:083
34611942112,6sleep30-21swapper/322:21:383
34561232112,6sleep30-21swapper/322:15:083
34413832112,6sleep30-21swapper/322:00:563
34271022112,5sleep30-21swapper/321:47:523
34067722112,5sleep30-21swapper/321:26:463
34019482112,6sleep30-21swapper/321:22:483
33967552112,6sleep30-21swapper/321:15:063
33918362112,6sleep30-21swapper/321:10:373
33918362112,6sleep30-21swapper/321:10:363
33811912112,5sleep30-21swapper/320:58:043
33719852112,5sleep30-21swapper/320:50:023
33423582112,6sleep30-21swapper/320:22:493
33373652112,6sleep30-21swapper/320:17:243
33284602112,5sleep30-21swapper/320:07:533
33226452112,5sleep30-21swapper/320:02:473
33175842112,6sleep30-21swapper/319:56:453
33029412112,5sleep30-21swapper/319:42:493
32877682112,6sleep30-21swapper/319:25:323
32874502113,5sleep30-21swapper/319:24:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional