You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-21 - 12:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Mar 21, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34143092940,3chrt75-21kworker/3:1+events22:01:433
33144772780,3sleep00-21swapper/020:21:400
34949282650,4chrt0-21swapper/123:26:321
33272712560,4chrt0-21swapper/120:36:381
33319392540,4sleep0201rcuc/020:41:390
3239230995449,4cyclictest3404945-21kworker/u8:1+events_unbound21:56:371
3239228995141,7cyclictest3275942-21latency_hist19:45:010
3239230994943,4cyclictest3494313-21kworker/u8:2+events_unbound00:11:361
3239228994739,5cyclictest3295924-21latency_hist20:05:010
3239228994737,7cyclictest3239077-21latency_hist19:10:000
3239230994541,3cyclictest3321110-21kworker/u8:2+events_unbound22:03:071
3239228994536,7cyclictest3365398-21latency_hist21:15:010
3239230994339,3cyclictest3239382-21kworker/u8:0+events_unbound19:46:361
3239230994339,3cyclictest3234197-21kworker/u8:2+events_unbound19:31:481
3239230994336,5cyclictest3260834-21latency_hist19:30:011
3239228994314,16cyclictest589-21runrttasks20:56:300
3239234994234,6cyclictest3260940-21gpgv19:30:012
3239230994238,3cyclictest3494313-21kworker/u8:2+flush-8:000:01:391
3239230994237,4cyclictest3379693-21kworker/u8:0+events_unbound21:33:541
3239228994235,5cyclictest3415131-21latency_hist22:05:010
3239228994210,10cyclictest589-21runrttasks22:26:310
3239228994210,10cyclictest589-21runrttasks22:26:310
34576772410,1sleep03457680-21cut22:46:380
3239230994137,3cyclictest3444864-21kworker/u8:1+flush-8:022:46:491
3239230994137,3cyclictest3419768-21kworker/u8:3+events_unbound22:08:571
3239230994137,3cyclictest3321110-21kworker/u8:2+events_unbound21:06:371
3239230994131,7cyclictest280-21systemd-journal21:26:541
3239230994036,3cyclictest3295946-21kworker/u8:1+events_unbound20:19:551
3239230994035,4cyclictest3321110-21kworker/u8:2+events_unbound21:11:481
3239230994031,7cyclictest280-21systemd-journal20:01:541
3239228994033,5cyclictest3455051-21latency_hist22:45:020
3239230993935,3cyclictest3365529-21kworker/u8:3+events_unbound21:23:011
3239230993935,3cyclictest3261344-21kworker/u8:1+events_unbound19:41:551
3239230993935,3cyclictest3234197-21kworker/u8:2+events_unbound19:15:551
3239230993934,4cyclictest3321110-21kworker/u8:2+events_unbound22:50:551
3239230993931,6cyclictest280-21systemd-journal20:10:001
3239228993930,6cyclictest3554231-21latency_hist00:25:000
3239230993835,2cyclictest3295946-21kworker/u8:1+events_unbound20:11:361
3239230993834,3cyclictest3494313-21kworker/u8:2+events_unbound23:56:481
3239230993834,3cyclictest3494313-21kworker/u8:2+events_unbound23:56:481
3239230993834,3cyclictest3469885-21kworker/u8:3+events_unbound23:15:031
3239230993834,3cyclictest3295946-21kworker/u8:1+events_unbound20:22:521
3239230993832,4cyclictest3239382-21kworker/u8:0+events_unbound20:51:471
3239230993832,4cyclictest3239382-21kworker/u8:0+events_unbound20:51:471
3239228993829,7cyclictest3400619-21/usr/sbin/munin21:51:400
3239234993733,3cyclictest3295946-21kworker/u8:1+events_unbound20:25:012
3239230993733,3cyclictest3261344-21kworker/u8:1+events_unbound19:36:501
3239230993733,3cyclictest3234197-21kworker/u8:2+events_unbound19:13:541
3239230993732,4cyclictest3469885-21kworker/u8:3+events_unbound00:31:451
323922899378,13cyclictest589-21runrttasks23:31:320
3239228993730,4cyclictest3514738-21apt-get23:46:300
3239228993728,6cyclictest280-21systemd-journal23:38:540
3239228993711,11cyclictest589-21runrttasks21:21:300
3239228993710,16cyclictest3565403-21/usr/sbin/munin00:36:400
3239228993710,11cyclictest3405931-21grep21:56:370
3239230993632,3cyclictest3524478-21kworker/u8:0+events_unbound00:35:541
3239230993631,4cyclictest3469885-21kworker/u8:3+events_unbound00:23:561
323922899369,10cyclictest589-21runrttasks00:31:300
323922899368,17cyclictest3395759-21aten2_r5power_e21:46:320
323922899368,12cyclictest3446186-21ps22:36:370
3239228993626,8cyclictest3280017-21ntpq19:46:420
3239228993611,13cyclictest589-21runrttasks22:06:330
3239228993610,13cyclictest3265229-21ntpq19:31:430
3239228993610,11cyclictest3529596-21apt-get00:01:320
3239230993531,3cyclictest3513624-21kworker/u8:1+events_unbound23:48:551
3239230993531,3cyclictest3444864-21kworker/u8:1+events_unbound23:01:501
3239230993531,3cyclictest3419768-21kworker/u8:3+events_unbound22:20:111
3239230993531,3cyclictest3419768-21kworker/u8:3+events_unbound22:20:111
323922899359,15cyclictest3310797-21munin-run20:20:000
323922899359,13cyclictest3465759-21/usr/sbin/munin22:56:370
323922899358,15cyclictest3350964-21aten2_r5power_e21:01:320
323922899357,15cyclictest3545562-21/usr/sbin/munin00:16:370
323922899355,19cyclictest3385846-21aten2_r5power_e21:36:310
3239228993529,3cyclictest3480835-21irqrtprio23:11:360
32389422354,10sleep10-21swapper/119:08:271
33371372340,6sleep30-21swapper/320:46:393
3239234993429,4cyclictest3524478-21kworker/u8:0+flush-8:000:21:452
3239234993427,5cyclictest280-21systemd-journal20:37:552
3239230993430,3cyclictest3444864-21kworker/u8:1+events_unbound22:37:541
3239230993430,3cyclictest3419768-21kworker/u8:3+events_unbound22:28:561
3239230993430,3cyclictest3419768-21kworker/u8:3+events_unbound22:28:561
3239230993430,3cyclictest3365529-21kworker/u8:3+events_unbound21:52:541
3239230993429,4cyclictest3509452-21systemd-run23:40:001
3239230993429,4cyclictest3494313-21kworker/u8:2+events_unbound23:24:541
3239230993429,3cyclictest3555309-21irqrtprio00:26:361
3239230993427,5cyclictest3425360-21/usr/sbin/munin22:16:421
323922899349,15cyclictest3441059-21/usr/sbin/munin22:31:370
323922899349,14cyclictest3394500-21ntpq21:41:440
323922899349,13cyclictest3366178-21/usr/sbin/munin21:16:320
323922899348,14cyclictest3339856-21ntpq20:46:450
323922899348,12cyclictest3344812-21ntpq20:51:440
323922899348,12cyclictest3344812-21ntpq20:51:440
323922899347,15cyclictest3489028-21sensors_volt23:16:460
323922899347,14cyclictest3266788-21hddtemp_smartct19:36:350
323922899347,14cyclictest3254941-21memory19:21:410
323922899347,14cyclictest0-21swapper/000:26:450
323922899346,16cyclictest3513891-21wc23:41:500
323922899345,17cyclictest0-21swapper/019:56:310
3239228993430,2cyclictest3430026-21systemd-run22:20:000
3239228993424,8cyclictest3376156-21diskstats21:26:340
3239228993420,5cyclictest0-21swapper/019:15:010
3239228993411,13cyclictest3495114-21/usr/sbin/munin23:26:330
3239228993411,11cyclictest3474023-21ntpq23:01:450
3239234993325,6cyclictest3440222-21cat22:30:002
3239234993325,6cyclictest3440222-21cat22:30:002
3239234993323,7cyclictest3275936-21turbostat.cron19:45:002
3239230993329,3cyclictest3365529-21kworker/u8:3+flush-8:021:46:551
3239230993328,4cyclictest3365529-21kworker/u8:3+events_unbound21:36:471
3239230993324,7cyclictest3420450-21apt-get22:11:311
323922899339,12cyclictest3434156-21/usr/sbin/munin22:21:430
323922899339,12cyclictest3434156-21/usr/sbin/munin22:21:430
323922899339,12cyclictest0-21swapper/023:51:300
323922899339,12cyclictest0-21swapper/023:51:300
323922899339,10cyclictest3534573-21apt-get00:06:310
323922899338,15cyclictest3301935-21irqrtprio20:11:390
323922899338,11cyclictest3250111-21ntpq19:16:420
323922899338,10cyclictest3528222-21memory23:56:420
323922899338,10cyclictest3528222-21memory23:56:420
323922899336,10cyclictest3424820-21taskset22:12:360
323922899335,5cyclictest3281818-21ps19:51:370
3239228993323,8cyclictest3325047-21ntpq20:31:450
3239234993224,7cyclictest3375535-21cron21:25:002
3239230993228,3cyclictest3239382-21kworker/u8:0+events_unbound20:40:011
3239230993228,3cyclictest3239382-21kworker/u8:0+events_unbound19:53:551
323922899326,9cyclictest3356602-21ps21:06:380
323922899326,14cyclictest3381258-21/usr/sbin/munin21:31:350
323922899326,10cyclictest3326912-21sed20:36:370
323922899325,7cyclictest3539429-21chrt00:11:050
323922899325,15cyclictest3475091-21sshd23:05:540
323922899325,15cyclictest3319695-21wc20:26:430
3239234993123,6cyclictest3549310-21awk00:20:002
3239230993124,5cyclictest3499879-21apt-get23:31:311
3239230993122,2cyclictest3545596-21kthreadcore00:16:381
323922899319,11cyclictest3493756-21ntpq23:21:440
323922899315,12cyclictest3463842-21/usr/sbin/munin22:51:400
323922899314,15cyclictest0-21swapper/020:06:300
3239228993112,9cyclictest3260054-21ntpq19:26:440
3239234993025,3cyclictest3419768-21kworker/u8:3+events_unbound22:20:062
3239234993025,3cyclictest3419768-21kworker/u8:3+events_unbound22:20:062
3239234993022,6cyclictest280-21systemd-journal22:51:542
3239230993026,3cyclictest3365529-21kworker/u8:3+events_unbound21:40:551
3239230993023,2cyclictest161rcu_preempt22:56:511
3239234992924,4cyclictest3404945-21kworker/u8:1+flush-8:022:02:002
3239234992923,5cyclictest1-21systemd00:00:012
3239234992923,5cyclictest1-21systemd00:00:002
3239234992918,9cyclictest3244337-21cat19:15:012
323923499291,10cyclictest34576802cut22:46:382
3239230992920,3cyclictest3321110-21kworker/u8:2+events_unbound22:43:021
33623582280,5sleep3481ktimers/321:11:393
3239234992824,3cyclictest3365529-21kworker/u8:3+flush-8:021:46:352
3239234992824,2cyclictest3529823-21cpuspeed_turbos00:01:342
3239234992824,2cyclictest3281262-21aten2_r5power_c19:51:322
3239234992823,4cyclictest3234197-21kworker/u8:2+flush-8:019:31:352
3239230992824,3cyclictest3239382-21kworker/u8:0+events_unbound20:29:541
3239230992824,3cyclictest3196423-21kworker/u8:3+events_unbound19:22:181
3239230992823,4cyclictest3469885-21kworker/u8:3+events_unbound23:11:401
3239230992822,4cyclictest3350541-21idleruntime-cro21:00:001
3239234992723,2cyclictest3345256-21unixbench_multi20:51:502
3239234992723,2cyclictest3345256-21unixbench_multi20:51:502
3239230992723,3cyclictest3444864-21kworker/u8:1+events_unbound23:08:151
3239230992721,4cyclictest3440537-21apt-get22:31:301
323923499266,2cyclictest34887041grep23:16:402
3239234992623,2cyclictest3286478-21fschecks_time19:56:352
3239234992621,3cyclictest3376124-21cpuspeed_turbos21:26:332
3239234992620,4cyclictest3394995-21unixbench_multi21:41:522
3239230992622,3cyclictest3430231-21kworker/u8:0+events_unbound23:40:401
3239230992621,3cyclictest3523358-21needreboot23:51:421
3239230992621,3cyclictest3523358-21needreboot23:51:421
33133322250,2sleep3481ktimers/320:21:393
3239234992521,3cyclictest3469885-21kworker/u8:3+events_unbound23:21:422
3239234992521,2cyclictest3325483-21unixbench_multi20:31:502
3239234992519,4cyclictest3479399-21unixbench_multi23:06:492
3239230992521,3cyclictest3524478-21kworker/u8:0+events_unbound00:05:081
3239230992521,3cyclictest3321110-21kworker/u8:2+events_unbound21:00:011
3239234992421,2cyclictest3301571-21fschecks_time20:11:362
3239234992420,3cyclictest3239382-21kworker/u8:0+events_unbound20:05:002
3239234992420,2cyclictest3484331-21grep23:11:482
3239234992419,3cyclictest3429970-21cron22:20:002
3239230992420,3cyclictest3321110-21kworker/u8:2+events_unbound20:46:121
3239230992418,4cyclictest3286165-21apt-get19:56:301
3239230992415,2cyclictest161rcu_preempt20:31:371
3239234992320,2cyclictest3276614-21fschecks_time19:46:352
3239234992319,3cyclictest3539116-21idleruntime-cro00:10:002
3239234992319,2cyclictest3523917-21grep23:51:502
3239234992319,2cyclictest3523917-21grep23:51:502
3239234992319,2cyclictest3445785-21fschecks_time22:36:342
3239234992319,2cyclictest3351152-21cpuspeed_turbos21:01:332
3239234992318,3cyclictest3454733-21unixbench_singl22:41:502
3239234992318,3cyclictest0-21swapper/223:45:002
3239230992312,1cyclictest421rcuog/221:19:391
3239234992219,2cyclictest3316397-21fschecks_time20:26:352
323923499217,9cyclictest371rcuc/219:16:362
3239234992117,3cyclictest3524478-21kworker/u8:0+events_unbound00:10:072
3239234992117,3cyclictest3415524-21aten2_r5power_c22:06:332
3239234992116,3cyclictest0-21swapper/221:15:012
3239234992112,7cyclictest3296551-21fschecks_count20:06:352
323923499205,2cyclictest34037461kthreadcore21:51:392
323923499204,4cyclictest0-21swapper/223:31:422
3239234992016,2cyclictest0-21swapper/223:36:502
3239234992014,4cyclictest3266496-21aten2_r5power_p19:36:322
32388072204,9sleep20-21swapper/219:06:522
323923499194,9cyclictest0-21swapper/219:21:352
323923499193,4cyclictest0-21swapper/222:11:452
323923499192,12cyclictest3470228-21apt-get23:01:322
323923499192,12cyclictest3365993-21apt-get21:16:312
3239234991916,2cyclictest3440887-21fschecks_time22:31:352
3239234991916,2cyclictest3306372-21cpuspeed_turbos20:16:332
3239234991915,2cyclictest0-21swapper/200:40:012
3239234991915,2cyclictest0-21swapper/200:31:342
3239234991914,3cyclictest0-21swapper/221:36:332
323923499182,4cyclictest0-21swapper/220:58:242
3239234991815,2cyclictest3405897-21fschecks_time21:56:372
3239234991814,3cyclictest3321110-21kworker/u8:2+flush-8:022:55:062
3239234991814,2cyclictest3381085-21fschecks_count21:31:342
323923499181,2cyclictest161rcu_preempt20:46:492
32344052183,8sleep30-21swapper/319:05:043
323923499173,4cyclictest0-21swapper/221:06:442
323923499173,3cyclictest0-21swapper/223:26:452
323923499172,4cyclictest0-21swapper/223:46:312
3239234991713,2cyclictest3331374-21tr20:41:342
323923499163,4cyclictest0-21swapper/200:26:422
33659672152,8sleep30-21swapper/321:16:303
35395292140,3sleep375-21kworker/3:1-mm_percpu_wq00:11:323
35046192132,6sleep30-21swapper/323:35:033
34995012132,7sleep30-21swapper/323:30:003
34815252131,8sleep30-21swapper/323:11:383
34042422132,7sleep30-21swapper/321:51:413
33313282130,3sleep3481ktimers/320:41:343
32660732132,6sleep30-21swapper/319:35:023
32654692132,7sleep30-21swapper/319:31:493
35657182122,6sleep30-21swapper/300:36:413
35496842122,6sleep30-21swapper/300:21:323
35361312122,7sleep30-21swapper/300:06:403
34932132122,6sleep30-21swapper/323:21:413
34895902122,5sleep30-21swapper/323:20:013
34756532122,6sleep30-21swapper/323:06:373
34704752121,7sleep30-21swapper/323:01:353
34625192122,6sleep30-21swapper/322:51:403
34566782122,6sleep30-21swapper/322:46:383
34472342122,6sleep30-21swapper/322:36:393
34445422122,6sleep30-21swapper/322:31:503
34151562122,5sleep30-21swapper/322:05:013
33806262122,6sleep30-21swapper/321:30:023
33763792122,6sleep30-21swapper/321:26:373
33556082122,6sleep30-21swapper/321:05:023
33540212121,7sleep30-21swapper/321:01:413
33481912122,6sleep30-21swapper/320:56:393
33258212122,5sleep30-21swapper/320:35:013
33157182122,6sleep30-21swapper/320:25:013
32971732122,6sleep30-21swapper/320:06:413
32953602123,6sleep30-21swapper/320:01:503
32717192122,6sleep30-21swapper/319:41:363
32514132122,6sleep30-21swapper/319:21:333
32397482122,6sleep30-21swapper/319:11:343
35595742112,5sleep30-21swapper/300:31:313
35555092112,5sleep30-21swapper/300:26:383
35444152112,5sleep30-21swapper/300:15:023
35336372112,5sleep30-21swapper/300:01:503
35245252113,5sleep30-21swapper/323:55:483
35245252113,5sleep30-21swapper/323:55:483
35205322112,5sleep30-21swapper/323:51:393
35205322112,5sleep30-21swapper/323:51:393
35143922112,5sleep30-21swapper/323:45:013
35100522112,5sleep30-21swapper/323:41:363
35020082112,5sleep30-21swapper/323:31:413
34677922112,6sleep30-21swapper/322:56:413
34504782112,5sleep30-21swapper/322:41:333
34352462112,5sleep30-21swapper/322:25:523
34352462112,5sleep30-21swapper/322:25:523
34302722112,5sleep30-21swapper/322:20:383
34302722112,5sleep30-21swapper/322:20:383
34277752112,5sleep30-21swapper/322:16:413
34247052112,5sleep30-21swapper/322:11:513
34054422112,5sleep30-21swapper/321:55:033
33960222112,5sleep30-21swapper/321:46:353
33904072112,5sleep30-21swapper/321:40:013
33901562112,6sleep30-21swapper/321:37:243
33722372112,5sleep30-21swapper/321:21:393
33407882112,5sleep30-21swapper/320:50:013
33407882112,5sleep30-21swapper/320:50:013
33262462112,5sleep30-21swapper/320:36:333
33060172112,5sleep30-21swapper/320:15:023
33015912112,5sleep30-21swapper/320:11:373
32861262112,6sleep30-21swapper/319:56:203
32810932112,5sleep30-21swapper/319:50:023
32763852112,5sleep30-21swapper/319:46:333
32566532112,5sleep30-21swapper/319:26:353
32501632112,5sleep30-21swapper/319:16:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional