You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 23:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun Mar 29, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3370155996642,20cyclictest3556366-21latency_hist10:15:010
3370155995332,18cyclictest3402069-21/usr/sbin/munin07:41:390
35166732510,3chrt75-21kworker/3:1+events09:36:263
3370161994838,8cyclictest280-21systemd-journal11:20:012
3370157994839,6cyclictest3391800-21latency_hist07:30:011
3370155994739,5cyclictest3586092-21latency_hist10:45:010
3370155994725,6cyclictest280-21systemd-journal07:29:480
3370155994625,6cyclictest280-21systemd-journal09:35:480
3370161994437,5cyclictest280-21systemd-journal11:46:472
3370155994437,5cyclictest3685130-21apt-get12:25:010
3370155994427,4cyclictest3416891-21apt-get07:56:230
337015599431,40cyclictest3461518-21unin-run08:40:010
35904322420,2sleep239-21ksoftirqd/210:46:442
3370157994031,6cyclictest3531435-21latency_hist09:50:011
3370157994031,6cyclictest3481690-21latency_hist09:00:011
3370157994010,17cyclictest3481947-21taskset09:00:151
3370155993925,10cyclictest3476520-21apt-get08:55:000
337015799387,18cyclictest3397379-21/usr/sbin/munin07:36:281
3370157993829,6cyclictest280-21systemd-journal09:29:481
3370155993829,6cyclictest3700101-21grep12:36:450
3370157993728,6cyclictest3471581-21latency_hist08:50:011
3370155993726,8cyclictest3445531-21ntpq08:21:370
337015799367,18cyclictest3460638-21ld08:36:391
337015799367,17cyclictest3575437-21ntp_states10:31:391
337015799367,17cyclictest3446881-21grep08:26:271
337015799366,16cyclictest280-21systemd-journal09:05:481
3370157993610,15cyclictest3595727-21taskset10:52:111
3370157993610,15cyclictest3595727-21taskset10:52:111
3370155993626,7cyclictest280-21systemd-journal12:10:480
337015599360,33cyclictest3605864-21true11:05:000
337015799358,16cyclictest3425578-21ntpq08:01:371
3370155993512,20cyclictest3571273-21kworker/0:0+events10:36:370
337015799349,12cyclictest3664583-21ntpq12:01:401
337015799349,12cyclictest3610017-21ntpq11:06:401
337015799348,12cyclictest3511725-21apt-get09:31:261
337015799348,12cyclictest3455700-21ntpq08:31:401
337015799347,15cyclictest3585229-21ntpq10:41:371
337015799346,15cyclictest3695819-21apt-get12:36:251
337015799346,15cyclictest3495973-21python309:11:441
3370157993430,3cyclictest3660450-21systemd-run12:00:001
3370157993427,5cyclictest3630574-21latency_hist11:30:011
3370157993425,6cyclictest3441440-21latency_hist08:20:011
3370157993424,7cyclictest280-21systemd-journal11:21:471
337015799339,12cyclictest3644577-21ntpq11:41:381
337015799339,12cyclictest3520466-21ntpq09:36:391
337015799338,13cyclictest3690318-21apt-get12:30:031
337015799338,12cyclictest3649515-21ntpq11:46:391
337015799338,12cyclictest3375291-21ntpq07:15:021
337015799337,14cyclictest3416556-21sh07:55:001
337015799337,14cyclictest3416556-21sh07:55:001
337015799337,12cyclictest3670653-21apt-get12:11:261
337015799337,12cyclictest3665717-21apt-get12:06:251
337015799337,11cyclictest3570497-21ntpq10:26:401
337015799336,9cyclictest3639712-21ntpq11:36:421
337015799336,17cyclictest3619917-21ntp_states11:16:401
337015799336,17cyclictest3545394-21awk10:01:351
337015799336,16cyclictest3532039-21diskstats09:51:281
337015799336,16cyclictest3466501-21runrttasks08:43:141
337015799336,12cyclictest3615000-21ld11:11:411
337015799335,19cyclictest885-21mta-sts-daemon10:18:491
3370157993316,6cyclictest3555466-21needreboot10:11:371
3370157993310,8cyclictest421rcuog/210:06:251
3370157993310,11cyclictest3525680-21ntpq09:41:401
337015599335,21cyclictest3381944-21kworker/0:0+events07:25:000
3370155993327,4cyclictest3501133-21unixbench_multi09:16:450
337015599332,29cyclictest3536391-21kworker/0:2+events10:21:330
3370161993222,8cyclictest280-21systemd-journal08:20:002
337015799328,14cyclictest0-21swapper/108:21:241
337015799328,12cyclictest3634713-21ntpq11:31:391
337015799328,12cyclictest3565539-21ntpq10:21:401
337015799327,8cyclictest3685457-21dpkg12:25:011
337015799327,15cyclictest3505616-21ntpq09:21:371
337015799327,14cyclictest3600071-21ntpq10:56:391
337015799327,14cyclictest3600071-21ntpq10:56:391
337015799327,12cyclictest3590219-21ntpq10:46:401
337015799327,12cyclictest3580309-21ntpq10:36:381
337015799327,12cyclictest3540478-21ntpq09:56:381
337015799327,12cyclictest3386034-21ntpq07:21:381
337015799326,9cyclictest3405814-21ntpq07:41:391
337015799326,14cyclictest0-21swapper/108:51:251
337015799325,15cyclictest3431697-21apt-get08:11:281
3370157993225,5cyclictest3496900-21apt-get09:16:251
337015599328,22cyclictest3501521-21kworker/0:1+events09:26:260
3370161993122,7cyclictest3521379-21cp09:40:002
337015799319,11cyclictest3690123-21taskset12:27:421
337015799318,11cyclictest3654666-21ntpq11:51:381
337015799317,12cyclictest3396012-21ntpq07:31:381
337015799316,7cyclictest3374170-21ntpq07:11:381
337015799316,14cyclictest3430489-21ntpq08:06:381
337015799316,13cyclictest3679359-21ntpq12:16:381
337015799315,10cyclictest3420453-21cat07:56:341
3370157993121,8cyclictest280-21systemd-journal11:05:001
337015599313,26cyclictest3610816-21kworker/0:2+events11:18:370
3370155993126,3cyclictest3557643-21kthreadcore10:16:330
3370155993122,7cyclictest3609925-21ntpq11:06:370
337015599311,28cyclictest3381944-21kworker/0:0+events07:31:360
337015799306,12cyclictest3410811-21ntpq07:46:381
337015799306,12cyclictest3410811-21ntpq07:46:381
3370157993022,5cyclictest3369993-21latency_hist07:10:011
3370155993025,3cyclictest3615732-21cron11:15:000
3370155993024,4cyclictest3566538-21kworker/u8:1+flush-8:011:46:430
337015599302,27cyclictest3421506-21kworker/0:1+events08:41:330
337015599302,27cyclictest3421506-21kworker/0:1+events08:06:380
337015599302,26cyclictest3421506-21kworker/0:1+events08:30:000
3370155993021,6cyclictest3456485-21latency_hist08:35:010
3370155993010,17cyclictest3571273-21kworker/0:0+events10:35:010
336980523020,4sleep00-21swapper/007:07:480
3370161992925,2cyclictest3680862-21fschecks_time12:21:292
3370161992925,2cyclictest3381926-21sendmail07:20:002
3370155992923,4cyclictest3422056-21cut08:01:270
3370155992919,8cyclictest3400857-21ntpq07:36:380
3370161992822,3cyclictest3676302-21irqrtprio12:16:322
3370155992821,4cyclictest280-21systemd-journal10:30:000
3370155992820,6cyclictest3415690-21ntpq07:51:390
3370155992820,6cyclictest3415690-21ntpq07:51:390
3370155992817,9cyclictest3470727-21ntpq08:46:370
3370155992811,15cyclictest3421506-21kworker/0:1+events08:16:240
3370155992810,16cyclictest3660464-21kworker/0:0+events12:16:260
3370155992810,16cyclictest3501521-21kworker/0:1+events09:51:250
36256492270,6sleep20-21swapper/211:25:012
3370161992723,3cyclictest3506903-21aten2_r5power_v09:26:272
3370161992723,2cyclictest3601472-21cpuspeed_turbos11:01:282
3370161992723,2cyclictest3515917-21unixbench_multi09:31:442
3370161992718,7cyclictest280-21systemd-journal09:55:002
337015599279,16cyclictest3536391-21kworker/0:2+events10:06:280
337015599279,16cyclictest3421506-21kworker/0:1+events09:01:330
337015599278,17cyclictest3481747-21kworker/0:0+events09:06:240
337015599277,18cyclictest3571273-21kworker/0:0+events10:53:580
337015599277,18cyclictest3571273-21kworker/0:0+events10:53:580
337015599274,19cyclictest3571273-21kworker/0:0+events10:46:260
3370155992722,3cyclictest3662947-21kthreadcore12:01:350
3370155992722,3cyclictest3655485-21cron11:54:590
3370155992720,5cyclictest280-21systemd-journal12:21:140
3370161992623,2cyclictest3402207-21aten2_r5power_v07:41:292
3370161992621,4cyclictest3610783-21idleruntime-cro11:10:012
337016199260,24cyclictest3471287-21unixbench_singl08:46:452
337015599269,15cyclictest3610816-21kworker/0:2+events11:36:420
337015599269,15cyclictest3501521-21kworker/0:1+events09:41:260
337015599268,16cyclictest3333295-21kworker/0:1+events07:16:420
337015599264,20cyclictest3640506-21kworker/0:1+events11:56:380
337015599262,16cyclictest3665722-21chrt12:06:240
3370155992621,3cyclictest401-21dbus-daemon12:35:000
3370155992620,4cyclictest3625303-21unixbench_multi11:21:440
3370155992620,4cyclictest3365327-21kworker/u8:1+events_unbound08:15:010
3370161992521,2cyclictest3689998-21unixbench_singl12:26:442
3370161992520,3cyclictest3636156-21fschecks_count11:36:302
3370161992518,5cyclictest280-21systemd-journal12:00:002
3370161992516,7cyclictest3436275-21apt-get08:15:002
3370161992516,7cyclictest3411963-21apt-get07:51:262
3370161992516,7cyclictest3411963-21apt-get07:51:262
337015599259,14cyclictest3397054-21kworker/0:2+events07:46:340
337015599259,14cyclictest3397054-21kworker/0:2+events07:46:340
337015599257,16cyclictest3501521-21kworker/0:1+events09:31:340
3370155992519,4cyclictest3545966-21unixbench_singl10:01:430
3370155992515,8cyclictest3540487-21ntpq09:56:380
3370155992514,7cyclictest3375007-21cron07:15:000
3370161992420,2cyclictest3446057-21unixbench_singl08:21:442
3370161992418,4cyclictest3600515-21unixbench_multi10:56:442
3370161992418,4cyclictest3600515-21unixbench_multi10:56:442
3370161992418,4cyclictest3461437-21apt-get08:40:002
3370155992418,3cyclictest0-21swapper/011:45:000
34474752230,3chrt75-21kworker/3:1+events08:26:323
3370161992319,3cyclictest3392466-21fschecks_time07:31:272
3370161992319,2cyclictest3501941-21aten2_r5power_p09:21:262
337015599238,13cyclictest3571273-21kworker/0:0+events10:56:240
337015599238,13cyclictest3571273-21kworker/0:0+events10:56:240
337015599235,16cyclictest3501521-21kworker/0:1+events09:21:250
337015599235,16cyclictest3421506-21kworker/0:1+events08:56:400
337015599233,19cyclictest3481747-21kworker/0:0+events09:11:340
337016199229,9cyclictest3373231-21kthreadcore07:11:352
3370161992218,2cyclictest3421129-21unixbench_singl07:56:432
3370155992218,3cyclictest3610503-21kworker/u8:2+events_unbound11:27:230
35474572210,2sleep2381ktimers/210:06:312
3370161992118,2cyclictest3661010-21cpuspeed_turbos12:01:282
3370161992118,2cyclictest3472255-21fschecks_time08:51:282
3370161992118,2cyclictest3397492-21fschecks_time07:36:302
3370161992117,2cyclictest0-21swapper/208:30:012
3370161992116,3cyclictest0-21swapper/208:05:012
3370161992115,4cyclictest3674904-21unixbench_multi12:11:452
3370161992114,5cyclictest3569180-21tr10:26:332
337015599215,14cyclictest3610816-21kworker/0:2+events11:30:480
3370155992117,3cyclictest3527425-21sort09:46:320
34559792200,6sleep20-21swapper/208:31:442
3370161992014,3cyclictest3615743-21sh11:15:002
33698092204,12sleep20-21swapper/207:07:502
337016199192,3cyclictest35657782threads10:21:432
3370161991916,2cyclictest3576843-21fschecks_count10:36:282
337016199190,5cyclictest3529143-21cut09:46:352
337016199190,3cyclictest36279031kthreadcore11:26:342
33654532193,9sleep30-21swapper/307:06:263
337016199183,10cyclictest3634748-21ntpq11:31:402
3370161991814,3cyclictest3360421-21kworker/u8:2+flush-8:007:46:332
3370161991814,3cyclictest3360421-21kworker/u8:2+flush-8:007:46:332
3370161991814,2cyclictest3581800-21fschecks_time10:41:292
3370161991814,2cyclictest3551897-21aten2_r5power_v10:11:272
3370161991814,2cyclictest0-21swapper/211:51:382
337016199181,12cyclictest3487022-21apt-get09:06:262
337016199180,5cyclictest36914882sed12:31:322
35917402170,3chrt0-21swapper/210:51:302
35917402170,3chrt0-21swapper/210:51:302
35016572170,3chrt75-21kworker/3:1-events09:20:013
337016199179,6cyclictest3483091-21tr09:01:332
337016199172,10cyclictest3541702-21apt-get10:01:252
3370161991713,2cyclictest0-21swapper/212:06:252
3370161991713,2cyclictest0-21swapper/208:56:322
3370161991713,2cyclictest0-21swapper/208:10:012
3370161991712,3cyclictest0-21swapper/212:36:342
3370161991712,3cyclictest0-21swapper/209:16:372
3370161991712,2cyclictest0-21swapper/207:26:282
337016199168,6cyclictest3495003-21tr09:11:352
3370161991613,2cyclictest0-21swapper/210:19:052
3370161991613,2cyclictest0-21swapper/209:41:072
3370161991612,2cyclictest0-21swapper/210:31:262
3370161991611,3cyclictest0-21swapper/209:56:352
337016199161,11cyclictest3465064-21kthreadcore08:41:362
337016199161,11cyclictest3382281-21apt-get07:21:252
35268382152,8sleep30-21swapper/309:46:273
3370161991510,3cyclictest0-21swapper/211:41:342
36358622142,8sleep30-21swapper/311:36:263
35376122144,6sleep30-21swapper/309:56:343
34163492146,5sleep30-21swapper/307:52:463
34163492146,5sleep30-21swapper/307:52:463
36996832133,6sleep30-21swapper/312:36:423
36107992132,6sleep30-21swapper/311:10:023
34264432132,6sleep30-21swapper/308:05:023
36913632122,6sleep30-21swapper/312:31:323
36703102122,5sleep30-21swapper/312:10:013
36606702123,5sleep30-21swapper/312:00:183
36408862122,6sleep30-21swapper/311:41:283
36349562122,7sleep30-21swapper/311:31:453
36165682122,6sleep30-21swapper/311:16:323
35958492122,6sleep30-21swapper/310:53:483
35958492122,6sleep30-21swapper/310:53:483
35905752122,6sleep30-21swapper/310:46:463
35814092122,6sleep30-21swapper/310:40:483
35614112122,5sleep30-21swapper/310:20:013
35571982122,6sleep30-21swapper/310:16:323
35479302122,6sleep30-21swapper/310:06:333
35322982122,6sleep30-21swapper/309:51:323
35113832122,5sleep30-21swapper/309:30:013
35107912122,6sleep30-21swapper/309:26:443
35016572122,5sleep30-21swapper/309:20:023
34855262122,6sleep30-21swapper/309:01:363
34766422121,6sleep30-21swapper/308:55:023
34660712122,6sleep30-21swapper/308:41:463
34568432122,6sleep30-21swapper/308:36:283
34412972122,6sleep30-21swapper/308:18:183
34319832122,6sleep30-21swapper/308:11:323
34174172122,6sleep30-21swapper/307:56:313
34011332122,6sleep30-21swapper/307:36:443
33750602122,5sleep30-21swapper/307:15:013
33732302122,6sleep30-21swapper/307:11:363
36856102112,5sleep30-21swapper/312:25:263
36846902112,5sleep30-21swapper/312:21:453
36760452112,5sleep30-21swapper/312:16:313
36695552112,5sleep30-21swapper/312:06:403
36526912112,5sleep30-21swapper/311:51:353
36457502112,5sleep30-21swapper/311:46:263
36297582112,5sleep30-21swapper/311:26:393
36253652112,5sleep30-21swapper/311:21:463
36061032112,6sleep30-21swapper/311:05:283
36015352112,5sleep30-21swapper/311:01:313
36001272112,5sleep30-21swapper/310:56:413
36001272112,5sleep30-21swapper/310:56:413
35765852112,5sleep30-21swapper/310:36:273
35712222112,5sleep30-21swapper/310:30:013
35702512112,6sleep30-21swapper/310:26:363
35516832112,6sleep30-21swapper/310:10:303
35416762112,5sleep30-21swapper/310:01:233
35219102112,5sleep30-21swapper/309:41:293
34919452112,5sleep30-21swapper/309:11:273
34873932112,6sleep30-21swapper/309:06:303
34722322112,5sleep30-21swapper/308:51:293
34513652112,5sleep30-21swapper/308:30:023
34457702112,5sleep30-21swapper/308:21:433
34217902112,6sleep30-21swapper/308:01:163
34069982112,5sleep30-21swapper/307:46:243
34069982112,5sleep30-21swapper/307:46:243
34025642112,5sleep30-21swapper/307:41:343
33920892112,5sleep30-21swapper/307:30:543
33910562113,5sleep30-21swapper/307:26:413
33823682112,5sleep30-21swapper/307:21:273
36557222102,5sleep30-21swapper/311:55:043
34673912102,5sleep30-21swapper/308:46:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional