You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 01:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Wed Jul 08, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
344572723630,2sleep375-21kworker/3:1+events08:10:153
338160599212192,18cyclictest3578091-21apt-key10:25:020
338161199189184,3cyclictest0-21swapper/210:25:012
369977621530,2sleep3471rcuc/312:25:153
338160999128124,3cyclictest0-21swapper/110:25:011
34381472750,2chrt75-21kworker/3:1+events08:05:003
36118742620,2sleep239-21ksoftirqd/210:55:152
3381609996154,5cyclictest3482789-21kworker/u8:3+events_unbound10:05:241
3381609996055,4cyclictest2014292-21kworker/u8:1+events_unbound07:15:031
3381609996055,4cyclictest2014292-21kworker/u8:1+events_unbound07:15:031
36818212590,3sleep375-21kworker/3:1-events12:05:233
3381609995954,4cyclictest3482789-21kworker/u8:3+events_unbound10:00:011
3381609995752,4cyclictest3557816-21kworker/u8:1+events_unbound10:30:241
3381609995549,4cyclictest2014292-21kworker/u8:1+events_unbound07:45:221
3381609995546,7cyclictest1704890-21systemd-journal09:54:591
3381609995546,6cyclictest3447933-21latency_hist08:15:001
35817632540,6sleep3451irq_work/310:25:153
3381609995445,6cyclictest3413199-21latency_hist07:40:011
3381609995348,4cyclictest3233151-21kworker/u8:2+events_unbound08:40:201
3381609995345,6cyclictest3567366-21kworker/u8:2+flush-8:010:45:201
3381605995335,15cyclictest1704890-21systemd-journal11:53:060
36194042510,4chrt0-21swapper/011:05:130
3381609995146,4cyclictest3587497-21kworker/u8:0+events_unbound11:30:081
3381609995144,5cyclictest2014292-21kworker/u8:1+events_unbound07:30:231
3381609995143,6cyclictest3462535-21kworker/u8:0+flush-8:008:35:161
3381609995044,4cyclictest3587497-21kworker/u8:0+events_unbound12:30:081
3381609995044,4cyclictest3482789-21kworker/u8:3+flush-8:010:35:211
3381609995044,4cyclictest3233151-21kworker/u8:2+flush-8:007:10:321
3381609995044,4cyclictest3233151-21kworker/u8:2+flush-8:007:10:311
3381609995043,5cyclictest3656761-21kworker/u8:3+events_unbound12:05:211
3381609995041,7cyclictest1704890-21systemd-journal09:15:061
3381609994944,4cyclictest3482789-21kworker/u8:3+events_unbound08:55:201
3381609994942,5cyclictest3617549-21kworker/u8:2+flush-8:011:35:191
3381609994941,6cyclictest3422777-21kworker/u8:0+flush-8:007:50:111
3381609994937,8cyclictest1704890-21systemd-journal11:41:061
3381609994837,9cyclictest1704890-21systemd-journal10:20:001
3381605994810,33cyclictest3453191-21kworker/0:2+events08:40:060
3381609994741,4cyclictest3642835-21kworker/u8:1+events_unbound11:50:081
3381609994738,7cyclictest3587497-21kworker/u8:0+events_unbound12:10:231
3381605994731,14cyclictest1704890-21systemd-journal07:43:090
3381609994642,2cyclictest3233151-21kworker/u8:2+events_unbound08:50:241
3381609994640,5cyclictest3587497-21kworker/u8:0+events_unbound11:00:261
3381609994640,5cyclictest3587497-21kworker/u8:0+events_unbound11:00:261
3381609994640,4cyclictest3513380-21kworker/u8:1+events_unbound09:30:241
338160599461,42cyclictest1-21systemd11:35:010
3381609994539,5cyclictest3587497-21kworker/u8:0+flush-8:012:20:101
3381609994538,5cyclictest3482789-21kworker/u8:3+flush-8:009:45:371
338160599451,27cyclictest3527205-21ntp_states09:30:210
3381609994439,3cyclictest3587497-21kworker/u8:0+events_unbound11:20:011
3381609994438,4cyclictest3642835-21kworker/u8:1+events_unbound12:35:241
3381605994430,12cyclictest3428411-21aten2_r5power_p07:55:080
3381609994338,4cyclictest3482789-21kworker/u8:3+events_unbound10:00:231
3381609994338,3cyclictest3617549-21kworker/u8:2+events_unbound11:25:081
3381609994337,4cyclictest3642835-21kworker/u8:1+flush-8:012:25:251
3381609994234,5cyclictest1704890-21systemd-journal09:39:591
3381605994237,4cyclictest3233151-21kworker/u8:2+events_unbound09:35:140
3381605994225,15cyclictest3503021-21systemd-run09:10:010
35603902410,5sleep20-21swapper/210:05:152
34832682400,3sleep00-21swapper/008:50:000
3381611994030,7cyclictest1704890-21systemd-journal07:20:002
3381611994030,7cyclictest1704890-21systemd-journal07:19:592
3381611994010,14cyclictest3523367-21apt-get09:30:082
3381609994035,4cyclictest3482789-21kworker/u8:3+flush-8:009:00:221
3381609994035,3cyclictest3482789-21kworker/u8:3+flush-8:009:05:261
338160599402,32cyclictest3381608-21latency_hist07:10:020
338160599402,32cyclictest3381608-21latency_hist07:10:020
33810692404,10sleep10-21swapper/107:07:491
33810692404,10sleep10-21swapper/107:07:491
3381609993935,3cyclictest2014292-21kworker/u8:1+flush-8:008:30:181
3381609993934,4cyclictest2014292-21kworker/u8:1+events_unbound07:25:241
3381609993933,5cyclictest3233151-21kworker/u8:2+events_unbound09:10:221
3381609993933,4cyclictest2014292-21kworker/u8:1+flush-8:008:05:211
3381609993931,6cyclictest3482789-21kworker/u8:3+events_unbound09:40:251
3381605993929,8cyclictest3407240-21ntpq07:30:200
3381605993923,14cyclictest3603200-21latency_hist10:50:010
3381605993922,14cyclictest0-21swapper/011:40:170
3381609993833,4cyclictest3622593-21kworker/u8:3+events_unbound11:20:191
3381609993833,4cyclictest3233151-21kworker/u8:2+events_unbound07:20:191
3381609993832,4cyclictest3587497-21kworker/u8:0+flush-8:011:45:211
338160599380,25cyclictest589-21runrttasks10:44:500
3381611993720,12cyclictest3563000-21apt-get10:10:072
3381609993732,3cyclictest3642835-21kworker/u8:1+flush-8:011:55:151
3381609993732,3cyclictest3513380-21kworker/u8:1+events_unbound09:20:191
3381609993731,4cyclictest3567366-21kworker/u8:2+flush-8:010:20:081
3381605993720,15cyclictest3460760-21kthreadcore08:25:140
3381605993720,13cyclictest3543899-21tr09:50:140
3381605993717,18cyclictest3472549-21unixbench_multi08:35:240
338160599371,25cyclictest3447042-21ntpq08:10:190
338160599370,25cyclictest3627555-21runrttasks11:11:210
338160599370,23cyclictest589-21runrttasks09:48:590
338160599370,23cyclictest3588224-21tr10:35:060
338161199369,11cyclictest3487167-21ntpq08:50:192
3381611993620,10cyclictest3623124-21apt-get11:10:072
3381609993631,4cyclictest3557816-21kworker/u8:1+events_unbound10:10:221
3381609993630,5cyclictest3587497-21kworker/u8:0+flush-8:011:05:181
3381609993627,7cyclictest3452176-21ntpq08:15:191
3381605993627,6cyclictest3458111-21latency_hist08:25:010
338160599362,22cyclictest0-21swapper/012:32:090
3381605993615,18cyclictest3562762-21true10:09:590
338160599360,25cyclictest35568842collect210:00:190
338161199359,9cyclictest3414062-21ps07:40:122
3381611993514,7cyclictest3667621-21sh11:55:012
338160599354,24cyclictest0-21swapper/012:20:060
338160599354,23cyclictest0-21swapper/011:55:040
338160599353,25cyclictest0-21swapper/011:45:070
338160599353,24cyclictest0-21swapper/007:45:080
3381605993527,6cyclictest1704890-21systemd-journal08:05:010
338160599352,23cyclictest0-21swapper/011:24:260
3381605993519,14cyclictest0-21swapper/012:35:170
338160599351,26cyclictest3448424-21apt-get08:15:070
338160599350,21cyclictest3587706-21taskset10:33:140
338161199348,16cyclictest3571868-21ntpq10:15:222
3381611993410,10cyclictest3603279-21apt-get10:50:062
338160599344,24cyclictest0-21swapper/008:05:040
338160599344,23cyclictest0-21swapper/011:35:050
338160599343,26cyclictest0-21swapper/007:20:070
338160599343,24cyclictest0-21swapper/011:15:050
338160599342,29cyclictest3548127-21apt-get09:55:050
338160599342,25cyclictest3672853-21apt-get12:00:060
3381605993417,15cyclictest3641359-21kthreadcore11:25:160
3381605993417,14cyclictest0-21swapper/010:15:080
338160599341,24cyclictest3701439-21ntpq12:25:200
3381605993411,22cyclictest3381407-21kworker/0:0+events07:15:050
3381605993411,22cyclictest3381407-21kworker/0:0+events07:15:040
338161199337,17cyclictest0-21swapper/212:07:492
338161199336,18cyclictest0-21swapper/207:34:432
338161199336,17cyclictest0-21swapper/209:41:532
3381611993317,10cyclictest3643069-21apt-get11:30:072
3381609993329,3cyclictest2014292-21kworker/u8:1+events_unbound07:55:071
3381609993329,2cyclictest3587497-21kworker/u8:0+events_unbound12:15:291
3381609993328,3cyclictest3482789-21kworker/u8:3+events_unbound10:40:071
3381609993328,3cyclictest2014292-21kworker/u8:1+events_unbound08:20:221
338160599333,23cyclictest0-21swapper/012:15:060
3381605993317,13cyclictest3423556-21cpuspeed07:50:080
338160599331,26cyclictest3493294-21apt-get09:00:070
338161199326,12cyclictest3702548-21apt-get12:30:052
3381611993212,6cyclictest3452167-21ntpq08:15:182
3381609993226,5cyclictest3642835-21kworker/u8:1+flush-8:012:00:211
3381609993224,6cyclictest2014292-21kworker/u8:1+flush-8:008:00:231
338160599325,19cyclictest0-21swapper/010:23:020
338160599323,23cyclictest0-21swapper/009:25:050
338160599323,23cyclictest0-21swapper/008:30:050
3381605993225,5cyclictest3567714-21cron10:15:000
338160599322,22cyclictest0-21swapper/009:42:130
3381605993215,15cyclictest0-21swapper/010:55:080
3381605993214,16cyclictest0-21swapper/012:10:120
338160599321,28cyclictest3613479-21diskstats11:00:100
338160599321,28cyclictest3613479-21diskstats11:00:100
338160599321,26cyclictest3398601-21apt-get07:25:070
3381605993212,17cyclictest1704890-21systemd-journal07:10:040
3381605993212,17cyclictest1704890-21systemd-journal07:10:030
338161199319,11cyclictest3676573-21ntpq12:00:172
338161199316,16cyclictest0-21swapper/212:21:062
338161199316,16cyclictest0-21swapper/209:37:352
338161199316,16cyclictest0-21swapper/209:10:552
338161199316,16cyclictest0-21swapper/208:38:532
338161199316,16cyclictest0-21swapper/207:22:432
3381611993122,7cyclictest3683010-21fschecks_count12:10:102
3381611993117,9cyclictest3588112-21apt-get10:35:052
3381611993112,6cyclictest3522217-21ntpq09:25:212
3381611993110,6cyclictest3513409-21apt-get09:20:072
3381609993127,3cyclictest2014292-21kworker/u8:1+flush-8:008:45:241
3381609993126,4cyclictest3617549-21kworker/u8:2+flush-8:011:10:251
338160599313,24cyclictest0-21swapper/009:20:080
3381605993117,12cyclictest3607987-21latency_hist10:55:000
3381605993114,14cyclictest0-21swapper/009:15:040
338161199306,9cyclictest0-21swapper/211:40:432
3381611993026,2cyclictest3593141-21aten2_r5power_c10:40:052
3381611993016,11cyclictest3618138-21/usr/sbin/munin11:05:042
338160599303,22cyclictest0-21swapper/007:35:060
338160599302,26cyclictest0-21swapper/009:10:060
338160599300,28cyclictest3487486-21/usr/sbin/munin08:50:260
338161199299,5cyclictest3617028-21ntpq11:00:222
338161199299,5cyclictest3617028-21ntpq11:00:212
338161199298,5cyclictest3488340-21apt-get08:55:072
3381611992925,2cyclictest3633140-21aten2_r5power_c11:20:072
3381611992925,2cyclictest3442515-21unixbench_multi08:05:252
338160599291,24cyclictest3677761-21munin-node12:05:050
338160599290,25cyclictest3488339-21/usr/sbin/munin08:55:060
338161199289,4cyclictest3437098-21ntpq08:00:192
338161199287,5cyclictest3453773-21perl08:20:102
3381611992825,2cyclictest3512708-21unixbench_singl09:15:232
3381611992824,2cyclictest3463378-21aten2_r5power_p08:30:052
3381611992822,4cyclictest3418700-21fschecks_count07:45:112
3381611992819,4cyclictest3472985-21apt-get08:40:032
3381611992817,9cyclictest3493586-21fschecks_count09:00:112
3381611992817,8cyclictest3447940-21munin-run08:15:012
3381611992815,10cyclictest3668397-21/usr/sbin/munin11:55:132
3381611992722,4cyclictest2014292-21kworker/u8:1+flush-8:008:25:212
3381611992722,3cyclictest3402792-21unixbench_multi07:25:252
3381611992721,4cyclictest3428639-21fschecks_time07:55:092
3381611992713,10cyclictest3709645-21kthreadcore12:35:132
3381609992721,3cyclictest0-21swapper/109:28:271
338161199269,5cyclictest3551903-21ntpq09:55:182
338161199262,15cyclictest0-21swapper/207:51:092
3381611992621,3cyclictest3598290-21cpuspeed_turbos10:45:082
3381611992619,5cyclictest3482789-21kworker/u8:3+flush-8:010:21:122
3381611992612,3cyclictest0-21swapper/211:26:542
3381611992612,10cyclictest3478889-21if_enp4s0.208:45:102
338161199261,16cyclictest0-21swapper/212:16:552
3381611992610,4cyclictest3556897-21ntpq10:00:212
3381609992621,4cyclictest2014292-21kworker/u8:1+flush-8:007:40:101
338161199257,5cyclictest3498251-21apt-get09:05:072
338161199254,9cyclictest0-21swapper/209:55:002
3381611992515,8cyclictest3661705-21/usr/sbin/munin11:45:182
3381611992515,7cyclictest3651855-21ntpq11:35:192
3381611992515,7cyclictest3542029-21ntpq09:45:362
3381611992511,10cyclictest3701934-21taskset12:25:292
3381611992414,8cyclictest3412593-21/usr/sbin/munin07:35:262
3381611992413,8cyclictest3630284-21ps11:15:142
3381611992412,8cyclictest3583194-21/usr/sbin/munin10:30:082
3381611992410,11cyclictest3386303-21seq07:11:532
3381611992410,11cyclictest3386303-21seq07:11:522
3381609992420,3cyclictest2014292-21kworker/u8:1+events_unbound08:28:171
3381609992419,3cyclictest0-21swapper/110:55:041
3381609992218,2cyclictest0-21swapper/110:50:221
3381611991910,7cyclictest3381621-21apt-config07:10:022
3381611991910,7cyclictest3381621-21apt-config07:10:012
36144572180,2chrt481ktimers/311:00:153
36144572180,2chrt481ktimers/311:00:143
33812032183,9sleep30-21swapper/307:09:313
33812032183,9sleep30-21swapper/307:09:313
36520242170,3sleep3481ktimers/311:35:233
34017392170,4chrt0-21swapper/307:25:153
35886082150,11sleep30-21swapper/310:35:123
34831352142,7sleep30-21swapper/308:50:013
36924962132,6sleep30-21swapper/312:20:013
36685902132,7sleep30-21swapper/311:55:143
35977132136,5sleep30-21swapper/310:43:513
35831332132,6sleep30-21swapper/310:30:013
35056732130,2sleep3471rcuc/309:10:153
34691412132,7sleep30-21swapper/308:35:113
34382792132,7sleep30-21swapper/308:05:063
34132462132,6sleep30-21swapper/307:40:023
34084532132,7sleep30-21swapper/307:35:093
33874002132,7sleep30-21swapper/307:15:063
33874002132,7sleep30-21swapper/307:15:053
37122212122,6sleep30-21swapper/312:40:003
37023112122,5sleep30-21swapper/312:30:023
36916872122,6sleep30-21swapper/312:15:253
36873852122,6sleep30-21swapper/312:15:013
36329362122,6sleep30-21swapper/311:20:033
36035292122,6sleep30-21swapper/310:50:093
36019642122,6sleep30-21swapper/310:45:223
35684552122,6sleep30-21swapper/310:15:123
35580262122,6sleep30-21swapper/310:05:063
35531412122,6sleep30-21swapper/310:00:083
35471152123,5sleep30-21swapper/309:50:253
35269612122,6sleep30-21swapper/309:30:183
35174632122,6sleep30-21swapper/309:20:263
34981272122,6sleep30-21swapper/309:05:013
34936702122,6sleep30-21swapper/309:00:133
34771052122,6sleep30-21swapper/308:40:183
34677622122,6sleep30-21swapper/308:32:073
34479532122,5sleep30-21swapper/308:15:013
34233612122,6sleep30-21swapper/307:50:063
34227552122,6sleep30-21swapper/307:45:273
36775192112,5sleep30-21swapper/312:05:003
36633932112,6sleep30-21swapper/311:50:123
36582142112,6sleep30-21swapper/311:45:103
36533132112,5sleep30-21swapper/311:40:093
36428162112,5sleep30-21swapper/311:30:013
36385602112,6sleep30-21swapper/311:25:123
36317402112,5sleep30-21swapper/311:15:163
36240132112,5sleep30-21swapper/311:10:143
36184232112,5sleep30-21swapper/311:05:103
35776952112,5sleep30-21swapper/310:25:003
35633712112,5sleep30-21swapper/310:10:113
35484392112,6sleep30-21swapper/309:55:103
35382172112,5sleep30-21swapper/309:45:073
35330502112,5sleep30-21swapper/309:40:013
35283082112,6sleep30-21swapper/309:35:073
35223972112,6sleep30-21swapper/309:25:253
35084272112,5sleep30-21swapper/309:15:053
34892272112,5sleep30-21swapper/308:55:153
34834892112,5sleep30-21swapper/308:50:083
34583332112,5sleep30-21swapper/308:25:053
34546512112,5sleep30-21swapper/308:20:143
34320602112,5sleep30-21swapper/307:55:193
34071032112,5sleep30-21swapper/307:30:193
33945972112,5sleep30-21swapper/307:20:163
33820652112,5sleep30-21swapper/307:10:163
33820652112,5sleep30-21swapper/307:10:163
36082102102,5sleep30-21swapper/310:55:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional