You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 13:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sun Aug 23, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
372531222630,5sleep3451irq_work/300:37:453
352221321870,4sleep30-21swapper/321:12:463
35671122800,2sleep3471rcuc/321:57:453
34101212750,7sleep3575-21kworker/3:2+mm_percpu_wq19:22:433
34345682540,3chrt0-21swapper/019:47:360
3397019995343,7cyclictest3697717-21turbostat.cron00:10:002
3397013995242,7cyclictest3697715-21latency_hist00:10:010
3397015994941,6cyclictest3672532-21latency_hist23:45:011
3397019994842,4cyclictest3677442-21apt-get23:50:002
3397013994843,3cyclictest3677932-21kworker/u8:1+flush-8:000:22:400
3397013994840,6cyclictest3703078-21/usr/sbin/munin00:17:540
3397013994839,6cyclictest3478761-21latency_hist20:30:010
3397013994838,6cyclictest275-21systemd-journal22:05:180
3397013994742,4cyclictest3365605-21kworker/u8:2+events_unbound19:15:030
3397013994741,4cyclictest3498390-21kworker/u8:2+flush-8:020:52:410
3397019994639,5cyclictest3365605-21kworker/u8:2+flush-8:020:22:522
36086292450,5sleep10-21swapper/122:42:371
3397015994541,3cyclictest3439258-21kworker/u8:1+events_unbound21:17:451
3397015994539,4cyclictest3439258-21kworker/u8:1+flush-8:023:17:411
3397015994538,5cyclictest3558832-21apt-get21:52:341
3397013994540,3cyclictest3439258-21kworker/u8:1+flush-8:020:17:520
3397015994439,4cyclictest3513668-21kworker/u8:3+flush-8:021:07:411
3397013994436,6cyclictest3483918-21kworker/u8:0+flush-8:020:37:390
3397015994334,6cyclictest3627796-21latency_hist23:00:011
3397015994334,6cyclictest3568542-21latency_hist22:00:011
3397013994337,5cyclictest3677932-21kworker/u8:1+events_unbound00:30:000
3397019994233,6cyclictest275-21systemd-journal20:00:002
33923332424,10sleep10-21swapper/119:07:081
3397015994136,4cyclictest3688056-21kworker/u8:0+flush-8:000:32:391
3397013994134,5cyclictest3688056-21kworker/u8:0+events_unbound00:40:000
3397013994134,5cyclictest3365605-21kworker/u8:2+flush-8:020:07:530
3397013994133,7cyclictest3401707-21kworker/u8:3+flush-8:020:42:550
3397019994031,7cyclictest3429123-21cat19:40:012
3397015994034,4cyclictest275-21systemd-journal21:44:181
3397015994033,5cyclictest3365605-21kworker/u8:2+flush-8:019:22:481
3397015994031,7cyclictest3512676-21ntpq21:02:491
3397015994031,6cyclictest3483673-21latency_hist20:35:011
3397015994030,7cyclictest3687982-21base6400:00:011
3397013994032,6cyclictest3401707-21kworker/u8:3+flush-8:020:47:540
3397013994032,6cyclictest3401707-21kworker/u8:3+flush-8:020:47:540
3397013994031,6cyclictest275-21systemd-journal23:05:180
3397015993933,4cyclictest3693232-21apt-get00:07:361
3397013993828,8cyclictest3651743-21ntpq23:22:460
339701399380,36cyclictest1-21systemd19:51:180
3397015993727,8cyclictest3587573-21ntpq22:17:481
3397013993733,3cyclictest3413599-21kworker/u8:0+flush-8:019:42:380
3397013993731,4cyclictest275-21systemd-journal22:55:000
3397013993728,7cyclictest275-21systemd-journal23:45:000
3397015993629,5cyclictest3534143-21apt-get21:27:361
3397013993629,5cyclictest3365605-21kworker/u8:2+flush-8:019:27:460
3397013993626,7cyclictest3508455-21latency_hist21:00:010
3397013993531,3cyclictest3423691-21cron19:35:000
3397019993429,4cyclictest3677932-21kworker/u8:1+flush-8:000:37:522
3397015993427,5cyclictest3662399-21cron23:35:001
3397015993426,6cyclictest3677443-21latency_hist23:50:011
3397015993426,6cyclictest3423695-21latency_hist19:35:011
3397015993425,7cyclictest3593249-21latency_hist22:25:011
3397015993425,7cyclictest3498980-21/usr/sbin/munin20:52:491
3397015993425,7cyclictest3365605-21kworker/u8:2+flush-8:020:22:481
3397013993426,6cyclictest275-21systemd-journal22:29:490
3397013993426,5cyclictest3538636-21latency_hist21:30:000
3397015993327,4cyclictest3439258-21kworker/u8:1+flush-8:020:57:521
3397015993327,4cyclictest3401707-21kworker/u8:3+flush-8:020:07:401
3397015993326,5cyclictest275-21systemd-journal22:13:181
3397013993327,4cyclictest3481342-21tail20:32:440
3397013993327,4cyclictest3401645-21grep19:12:550
3397013993321,10cyclictest3428659-21latency_hist19:40:010
3397019993227,3cyclictest3459321-21aten2_r5power_v20:12:382
3397015993222,8cyclictest3638111-21/usr/sbin/munin23:12:481
3397013993225,5cyclictest3439258-21kworker/u8:1+flush-8:021:32:520
339701999310,11cyclictest34231022cut19:32:522
3397015993127,3cyclictest3468635-21apt-get20:20:001
3397015993127,3cyclictest3401707-21kworker/u8:3+flush-8:019:45:361
3397015993123,6cyclictest3439454-21apt-get19:52:341
3397015993123,6cyclictest275-21systemd-journal21:25:001
3397013993126,4cyclictest3688056-21kworker/u8:0+flush-8:000:12:410
3397013993126,3cyclictest3563016-21kworker/u8:3+flush-8:022:47:570
3397019993025,3cyclictest3503493-21latency_hist20:55:002
3397019993010,3cyclictest36153972tr22:47:472
3397015993021,7cyclictest3397330-21/usr/sbin/munin19:12:471
3397013993026,3cyclictest3401707-21kworker/u8:3+flush-8:020:02:490
3397013993026,2cyclictest3656774-21kworker/u8:3+events_unbound23:30:010
3397013993024,5cyclictest3439258-21kworker/u8:1+events_unbound22:12:520
3397013993017,11cyclictest0-21swapper/023:47:430
3397019992924,3cyclictest3509265-21fschecks_time21:02:402
3397015992924,4cyclictest3543219-21kworker/u8:0+flush-8:022:02:421
3397015992924,3cyclictest3489728-21irqcore20:42:421
3397015992923,4cyclictest3523358-21unixbench_singl21:12:561
3397015992923,4cyclictest3478786-21latency_hist20:30:001
3397015992921,6cyclictest3401707-21kworker/u8:3+flush-8:020:47:551
3397015992921,6cyclictest3401707-21kworker/u8:3+flush-8:020:47:551
3397013992925,3cyclictest3563016-21kworker/u8:3+flush-8:022:03:480
3397013992925,3cyclictest3365605-21kworker/u8:2+flush-8:020:10:490
3397013992924,4cyclictest3598201-21kworker/u8:2+events_unbound22:42:500
3397013992924,3cyclictest3638371-21cut23:12:390
3397013992923,4cyclictest3553279-21unixbench_singl21:42:560
3397013992923,4cyclictest3523579-21kworker/u8:2+flush-8:021:52:390
3397019992824,3cyclictest3656774-21kworker/u8:3+flush-8:023:27:542
3397019992824,2cyclictest3569289-21fschecks_count22:02:382
3397019992823,3cyclictest3662141-21unixbench_multi23:32:542
3397019992817,8cyclictest3608092-21awk22:40:002
3397015992825,2cyclictest3650181-21kthreadcore23:22:431
3397015992824,3cyclictest3488415-21unixbench_multi20:37:541
3397015992823,3cyclictest3667567-21latency_hist23:40:011
3397015992819,7cyclictest3688247-21/usr/sbin/munin00:02:441
3397013992824,3cyclictest3533486-21systemd-run21:25:010
3397013992824,3cyclictest3439258-21kworker/u8:1+events_unbound23:19:120
3397013992822,4cyclictest3623561-21cpuspeed_turbos22:57:390
3397013992820,5cyclictest3687814-21latency_hist00:00:010
339701399280,26cyclictest3448920-21unin-run20:00:000
3397019992723,2cyclictest3648375-21fschecks_time23:22:382
3397019992723,2cyclictest3504226-21fschecks_count20:57:382
3397019992722,3cyclictest3434845-21fschecks_time19:47:382
3397019992722,3cyclictest3433590-21unixbench_singl19:42:562
3397019992718,7cyclictest3597623-21wc22:27:522
3397019992717,8cyclictest3478756-21awk20:30:002
3397019992712,2cyclictest0-21swapper/222:07:482
3397015992723,3cyclictest3599555-21kthreadcore22:32:431
3397015992723,3cyclictest3365605-21kworker/u8:2+flush-8:020:02:411
3397015992722,4cyclictest3688056-21kworker/u8:0+flush-8:000:37:401
3397015992720,5cyclictest3632745-21cron23:05:001
3397015992720,5cyclictest3632745-21cron23:05:001
3397015992718,7cyclictest3557738-21ntpq21:47:501
3397015992717,8cyclictest3428657-21apt-get19:40:001
3397015992715,9cyclictest275-21systemd-journal22:10:001
3397013992722,4cyclictest3563016-21kworker/u8:3+flush-8:023:02:540
3397013992722,4cyclictest3563016-21kworker/u8:3+flush-8:023:02:540
3397013992722,4cyclictest3523579-21kworker/u8:2+flush-8:021:47:390
3397013992721,4cyclictest3656774-21kworker/u8:3+flush-8:023:52:440
3397013992721,3cyclictest3543219-21kworker/u8:0+flush-8:021:57:390
3397013992720,5cyclictest3513668-21kworker/u8:3+flush-8:021:12:400
3397013992719,5cyclictest3663165-21cut23:37:390
33968322272,18sleep03396873-21systemd-run19:10:010
36448442260,3sleep3575-21kworker/3:2+events23:17:433
3397019992622,3cyclictest3543290-21unixbench_multi21:32:542
3397019992622,3cyclictest3498368-21unixbench_singl20:47:572
3397019992622,3cyclictest3498368-21unixbench_singl20:47:572
3397019992620,3cyclictest0-21swapper/222:45:002
3397019992610,5cyclictest0-21swapper/222:32:362
3397015992622,3cyclictest3657184-21unixbench_multi23:27:551
3397015992622,3cyclictest3413599-21kworker/u8:0+flush-8:019:42:401
3397015992621,3cyclictest3698224-21aten2_r5power_e00:12:381
3397015992618,6cyclictest275-21systemd-journal22:40:001
3397013992622,2cyclictest0-21swapper/023:32:350
3397013992621,4cyclictest3688056-21kworker/u8:0+flush-8:000:32:510
3397013992621,3cyclictest3589509-21irqrtprio22:22:420
3397013992621,3cyclictest3509179-21cut21:02:390
3397013992621,3cyclictest0-21swapper/021:20:000
3397013992620,4cyclictest3588339-21sendmail22:20:000
3397019992522,2cyclictest3493382-21unixbench_singl20:42:552
3397019992520,4cyclictest3365605-21kworker/u8:2+flush-8:019:15:062
3397019992520,3cyclictest3683104-21aten2_r5power_c23:57:362
3397019992518,5cyclictest3523558-21awk21:15:002
339701999250,6cyclictest36803752cut23:52:442
3397015992521,2cyclictest0-21swapper/120:12:561
3397015992520,4cyclictest3563016-21kworker/u8:3+flush-8:022:52:541
3397015992520,4cyclictest3563016-21kworker/u8:3+flush-8:022:47:561
3397015992519,4cyclictest275-21systemd-journal19:27:181
3397013992519,4cyclictest3563016-21kworker/u8:3+flush-8:022:37:530
3397013992517,6cyclictest3523579-21kworker/u8:2+flush-8:021:37:500
3397019992421,2cyclictest3663142-21fschecks_count23:37:392
3397019992420,3cyclictest3718139-21cpuspeed_turbos00:32:372
3397019992414,8cyclictest3468749-21cp20:20:002
339701999240,8cyclictest3706245-21cut00:17:462
3397015992419,3cyclictest3682434-21unixbench_singl23:52:551
3397015992419,3cyclictest0-21swapper/122:30:011
3397015992417,6cyclictest3408528-21unixbench_singl19:17:551
3397015992416,6cyclictest3444409-21apt-get19:57:341
3397013992419,3cyclictest0-21swapper/022:32:520
3397019992319,3cyclictest3439258-21kworker/u8:1+flush-8:020:37:512
3397019992318,3cyclictest3414550-21fschecks_time19:27:382
3397019992315,6cyclictest3643793-21ps23:17:412
3397015992319,3cyclictest3712350-21unixbench_singl00:22:551
3397015992318,2cyclictest0-21swapper/123:07:351
3397013992317,3cyclictest0-21swapper/021:07:370
33968042235,12sleep20-21swapper/219:09:382
339701999228,2cyclictest421rcuog/200:27:532
3397019992219,2cyclictest3668234-21cpuspeed_turbos23:42:362
3397019992217,3cyclictest3712530-21cron00:25:002
3397019992217,3cyclictest3622879-21cron22:55:002
3397015992217,3cyclictest0-21swapper/121:31:181
3397015992216,5cyclictest3439258-21kworker/u8:1+events_unbound21:37:491
3397015992216,4cyclictest3717239-21unixbench_multi00:27:561
339701999212,14cyclictest3409267-21apt-get19:22:372
339701999212,13cyclictest3534143-21apt-get21:27:372
3397019992117,3cyclictest3513884-21aten2_r5power_c21:07:382
3397019992114,5cyclictest3627812-21awk23:00:002
3397019992112,7cyclictest3548531-21/usr/sbin/munin21:40:012
339701999210,11cyclictest35328082ld21:22:512
3397015992112,7cyclictest849-21mta-sts-daemon00:17:401
3397013992118,2cyclictest3365605-21kworker/u8:2+events_unbound19:20:180
3397013992117,2cyclictest0-21swapper/000:02:330
3397013992116,4cyclictest3365605-21kworker/u8:2+events_unbound20:20:180
33921912217,8sleep30-21swapper/319:05:183
3397019992016,2cyclictest0-21swapper/221:55:012
3397019992016,2cyclictest0-21swapper/200:02:402
3397019992014,4cyclictest3701282-21kthreadcore00:12:462
3397019991915,2cyclictest0-21swapper/219:52:432
3397019991914,3cyclictest3549214-21cpuspeed_turbos21:42:382
3397019991814,2cyclictest0-21swapper/220:07:442
3397019991813,3cyclictest3564370-21cut21:57:392
3397019991812,5cyclictest3401864-21cron19:15:002
3397019991812,4cyclictest3588089-21unixbench_singl22:17:552
3397019991810,6cyclictest3630713-21tr23:02:452
3397019991810,6cyclictest3630713-21tr23:02:452
339701999178,7cyclictest3554211-21sort21:47:382
3397019991713,2cyclictest0-21swapper/221:17:372
3397019991713,2cyclictest0-21swapper/220:02:472
3397019991712,4cyclictest0-21swapper/220:32:532
3397019991711,5cyclictest3589854-21kthreadcore22:22:432
3397019991710,5cyclictest3636428-21tr23:07:452
36085442162,9sleep30-21swapper/322:42:363
36075942167,5sleep30-21swapper/322:37:553
339701999168,6cyclictest3640617-21kthreadcore23:12:462
3397019991613,2cyclictest0-21swapper/222:12:552
36830782132,7sleep30-21swapper/323:57:373
36777852133,6sleep30-21swapper/323:50:023
36623942132,6sleep30-21swapper/323:35:013
36396992134,6sleep30-21swapper/323:12:463
36029922132,7sleep30-21swapper/322:33:173
35535012132,6sleep30-21swapper/321:45:023
35339202132,7sleep30-21swapper/321:25:033
35285462132,6sleep30-21swapper/321:20:023
34330962134,5sleep30-21swapper/319:42:503
37217922122,6sleep30-21swapper/300:32:523
37028962122,6sleep30-21swapper/300:15:183
37024682122,6sleep30-21swapper/300:12:583
36975452122,6sleep30-21swapper/300:08:003
36921542122,6sleep30-21swapper/300:02:503
36718052122,6sleep30-21swapper/323:42:483
36525722122,5sleep30-21swapper/323:25:023
36130472122,5sleep30-21swapper/322:45:013
35974122122,6sleep30-21swapper/322:27:483
35876812122,6sleep30-21swapper/322:17:523
35784352122,6sleep30-21swapper/322:10:023
35739412123,6sleep30-21swapper/322:07:373
35703932120,2sleep3575-21kworker/3:2-mm_percpu_wq22:02:423
35588362122,6sleep30-21swapper/321:52:363
35439882122,7sleep30-21swapper/321:37:353
35390492122,6sleep30-21swapper/321:32:343
35251882122,6sleep30-21swapper/321:17:453
35182002122,6sleep30-21swapper/321:07:583
35091682122,6sleep30-21swapper/321:02:403
35075532123,5sleep30-21swapper/320:57:463
34968262122,6sleep30-21swapper/320:47:473
34968262122,6sleep30-21swapper/320:47:473
34891412122,6sleep30-21swapper/320:42:383
34856912122,6sleep30-21swapper/320:37:443
34796992122,6sleep30-21swapper/320:32:423
34765552122,5sleep30-21swapper/320:27:453
34737802122,6sleep30-21swapper/320:25:003
34591822122,7sleep30-21swapper/320:12:363
34538772122,5sleep30-21swapper/320:05:013
34394582122,6sleep30-21swapper/319:52:373
34241462122,6sleep30-21swapper/319:37:363
34189182122,6sleep30-21swapper/319:30:023
34069322122,6sleep30-21swapper/319:17:463
33986942122,6sleep30-21swapper/319:12:443
37125662112,5sleep30-21swapper/300:25:013
37080592112,5sleep30-21swapper/300:22:373
36729782112,5sleep30-21swapper/323:47:373
36577272112,6sleep30-21swapper/323:30:443
36485202112,6sleep30-21swapper/323:22:413
36331102112,6sleep30-21swapper/323:06:443
36281372112,6sleep30-21swapper/323:01:143
36281372112,6sleep30-21swapper/323:01:143
36235982112,5sleep30-21swapper/322:57:413
36187482112,5sleep30-21swapper/322:52:403
35887022112,6sleep30-21swapper/322:21:543
35487972112,5sleep30-21swapper/321:40:243
34987892112,5sleep30-21swapper/320:50:083
34681102112,6sleep30-21swapper/320:17:543
34517642111,6sleep30-21swapper/320:02:453
34442072112,6sleep30-21swapper/319:55:183
34360292112,6sleep30-21swapper/319:47:443
34141732112,6sleep30-21swapper/319:27:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional