You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 13:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Mon Apr 06, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40901732920,5sleep30-21swapper/300:31:283
38584262540,1sleep03858427-21grep20:36:290
3768632995043,5cyclictest3790474-21kworker/u8:3+flush-8:020:06:341
39570552490,2sleep3481ktimers/322:16:283
3768629994939,8cyclictest4013201-21ntpq23:11:310
3768632994739,6cyclictest280-21systemd-journal00:35:001
3768629994637,6cyclictest4098227-21latency_hist00:40:010
3768629994439,4cyclictest3953966-21kworker/u8:0+events_unbound23:51:270
3768632994231,8cyclictest280-21systemd-journal19:25:001
3768632994231,8cyclictest280-21systemd-journal19:25:001
3768629994134,5cyclictest4004549-21apt-get23:06:180
3768629994134,5cyclictest3824936-21grep20:01:380
3768636994024,14cyclictest3780348-21apt-get19:20:002
3768632994030,8cyclictest280-21systemd-journal21:40:011
3768629994031,6cyclictest3780356-21latency_hist19:20:000
3768636993924,13cyclictest3845701-21fschecks_count20:26:222
3768632993930,7cyclictest3983490-21ntpq22:41:351
376862999392,33cyclictest3790753-21kworker/0:2+events20:41:190
3768632993829,6cyclictest280-21systemd-journal00:06:361
3768632993728,6cyclictest3899743-21latency_hist21:20:001
3768629993732,4cyclictest3904688-21kworker/u8:1+flush-8:023:21:280
3768629993729,6cyclictest4059031-21apt-get00:01:190
3768629993728,7cyclictest4028039-21ntpq23:26:310
376862999372,30cyclictest3790753-21kworker/0:2+events19:55:040
37638342374,9sleep10-21swapper/119:05:251
37638342374,9sleep10-21swapper/119:05:251
3768632993627,6cyclictest3908825-21ntpq21:26:341
3768632993626,8cyclictest3978439-21ntpq22:36:321
3768632993623,5cyclictest3786144-21apt-get19:26:191
376862999362,29cyclictest3949421-21kworker/0:0+events22:30:070
3768636993525,8cyclictest4058663-21cp00:00:012
376862999353,30cyclictest3949421-21kworker/0:0+events22:17:380
376862999353,30cyclictest3894775-21kworker/0:1+events21:52:470
3768629993528,5cyclictest280-21systemd-journal22:21:360
3768629993430,3cyclictest4076026-21kthreadcore00:16:310
3768629993427,5cyclictest280-21systemd-journal20:47:040
3768629993424,7cyclictest3883778-21ntpq21:01:310
3768629993424,7cyclictest3883778-21ntpq21:01:310
376862999341,32cyclictest3894775-21kworker/0:1+events21:32:080
376862999341,28cyclictest3790753-21kworker/0:2+events20:05:490
376862999340,29cyclictest3894775-21kworker/0:1+events22:05:190
3768632993326,5cyclictest4004202-21latency_hist23:05:011
3768629993326,5cyclictest280-21systemd-journal20:27:000
376862999332,30cyclictest3894775-21kworker/0:1+events22:02:390
376862999332,30cyclictest3790753-21kworker/0:2+events20:56:560
376862999332,26cyclictest3894775-21kworker/0:1+events21:55:460
376862999332,26cyclictest3790753-21kworker/0:2+events20:20:070
376862999331,28cyclictest3790753-21kworker/0:2+events20:30:290
376862999331,27cyclictest3790753-21kworker/0:2+events19:45:520
376863699325,22cyclictest3905273-21cat21:26:242
376863699323,22cyclictest4049045-21apt-get23:51:202
3768636993215,15cyclictest3919520-21apt-get21:40:002
3768632993227,3cyclictest4010224-21irqrtprio23:11:261
3768632993224,2cyclictest161rcu_preempt21:25:011
3768632993222,8cyclictest3968569-21ntpq22:26:311
376862999328,20cyclictest3979349-21kworker/0:1+events23:00:080
376862999328,19cyclictest3949421-21kworker/0:0+events22:45:190
376862999328,19cyclictest3758730-21kworker/0:0+events19:11:060
376862999328,19cyclictest3758730-21kworker/0:0+events19:11:060
3768629993223,7cyclictest4072662-21ntpq00:11:310
376862999322,29cyclictest3790753-21kworker/0:2+events21:09:040
376862999321,30cyclictest3979349-21kworker/0:1+events22:54:340
376862999321,30cyclictest3949421-21kworker/0:0+events22:42:140
376862999321,30cyclictest3894775-21kworker/0:1+events21:39:380
376862999321,30cyclictest3790753-21kworker/0:2+events20:14:150
376862999321,27cyclictest3979349-21kworker/0:1+events23:35:250
376862999320,31cyclictest3894775-21kworker/0:1+events21:44:080
376863699313,21cyclictest4093613-21apt-get00:36:202
376863699312,25cyclictest4047940-21ntpq23:46:342
3768636993116,12cyclictest3796095-21aten2_r5power_e19:36:212
3768636993114,14cyclictest3984706-21aten2_r5power_v22:46:212
3768632993127,3cyclictest3951901-21kthreadcore22:11:291
3768632993124,5cyclictest3860289-21apt-get20:41:211
3768632993115,5cyclictest0-21swapper/119:41:371
3768629993123,4cyclictest0-21swapper/019:36:200
376862999312,28cyclictest3790753-21kworker/0:2+events19:52:450
3768629993121,7cyclictest3889580-21apt-get21:10:010
376862999311,29cyclictest3979349-21kworker/0:1+events23:19:220
376862999311,29cyclictest3979349-21kworker/0:1+events22:58:320
376862999310,30cyclictest3979349-21kworker/0:1+events00:20:150
3768636993026,3cyclictest3819461-21needreboot19:56:302
3768636993023,5cyclictest3773629-21kworker/u8:1+flush-8:019:26:342
3768636993015,12cyclictest4024418-21aten2_r5power_v23:26:202
3768632993026,2cyclictest3855503-21cpuspeed_turbos20:36:211
3768632993023,5cyclictest4068919-21apt-get00:11:181
3768632993022,2cyclictest161rcu_preempt20:31:231
3768632993020,8cyclictest3834401-21ntpq20:11:331
376862999302,27cyclictest3949421-21kworker/0:0+events22:29:180
376862999302,27cyclictest3894775-21kworker/0:1+events21:23:010
376862999302,24cyclictest3979349-21kworker/0:1+events23:41:110
376862999301,28cyclictest3790753-21kworker/0:2+events20:50:550
3768629993010,19cyclictest3790753-21kworker/0:2+events19:30:440
376863699298,18cyclictest3923155-21cut21:41:302
376863699292,5cyclictest3890166-21apt-get21:11:212
376863699292,23cyclictest4072721-21ntpq00:11:322
376863699292,21cyclictest3884996-21apt-get21:06:202
3768636992916,11cyclictest0-21swapper/223:15:002
3768636992916,11cyclictest0-21swapper/222:31:382
3768636992915,12cyclictest3910112-21aten2_r5power_v21:31:222
3768636992914,13cyclictest0-21swapper/220:56:342
3768636992912,14cyclictest3873983-21ntpq20:51:332
3768632992925,2cyclictest3970488-21irqrtprio22:31:251
3768632992925,2cyclictest3806651-21irqrtprio19:46:261
3768632992920,7cyclictest4093835-21cpuspeed_turbos00:36:221
3768632992918,2cyclictest161rcu_preempt22:16:221
3768632992913,3cyclictest0-21swapper/119:11:381
3768632992913,3cyclictest0-21swapper/119:11:381
376862999299,18cyclictest3790753-21kworker/0:2+events19:43:010
376862999295,19cyclictest3979349-21kworker/0:1+events00:25:120
3768629992924,3cyclictest3781188-21cpuspeed_turbos19:21:220
3768629992924,3cyclictest3781188-21cpuspeed_turbos19:21:220
376862999292,26cyclictest3979349-21kworker/0:1+events23:30:580
376862999292,26cyclictest3979349-21kworker/0:1+events23:30:580
376862999292,26cyclictest3758730-21kworker/0:0+events19:27:190
376862999292,22cyclictest3790753-21kworker/0:2+events21:15:510
376863699289,16cyclictest4043752-21cron23:45:012
3768636992824,2cyclictest3768949-21aten2_r5power_e19:11:222
3768636992824,2cyclictest3768949-21aten2_r5power_e19:11:222
376863699282,21cyclictest3943640-21ntpq22:01:312
376863699282,20cyclictest3864133-21ntpq20:41:372
3768636992815,12cyclictest3969060-21unixbench_multi22:26:392
3768636992815,11cyclictest0-21swapper/220:31:372
3768636992814,12cyclictest0-21swapper/220:21:322
3768636992813,13cyclictest0-21swapper/223:21:212
3768632992824,2cyclictest4009146-21cron23:10:001
3768632992822,4cyclictest3791080-21apt-get19:31:191
3768632992821,2cyclictest161rcu_preempt21:31:331
3768632992821,2cyclictest161rcu_preempt21:05:001
3768632992821,2cyclictest161rcu_preempt21:05:001
3768632992818,3cyclictest161rcu_preempt20:51:271
3768632992817,4cyclictest161rcu_preempt20:01:221
3768632992815,8cyclictest0-21swapper/120:46:321
3768632992815,7cyclictest0-21swapper/121:56:331
3768632992815,7cyclictest0-21swapper/120:21:361
3768632992815,7cyclictest0-21swapper/100:21:341
3768632992815,2cyclictest161rcu_preempt21:06:241
3768632992813,4cyclictest0-21swapper/120:26:341
376862999286,21cyclictest3979349-21kworker/0:1+events00:08:580
376862999282,25cyclictest3949421-21kworker/0:0+events22:10:390
376862999282,25cyclictest3790753-21kworker/0:2+events20:16:240
376862999281,26cyclictest3979349-21kworker/0:1+events23:55:110
376862999281,26cyclictest3979349-21kworker/0:1+events23:47:330
376862999281,26cyclictest3979349-21kworker/0:1+events00:31:110
376862999281,26cyclictest3949421-21kworker/0:0+events22:39:190
376862999281,26cyclictest3894775-21kworker/0:1+events21:49:360
37683942283,4sleep00-21swapper/019:09:050
37683942283,4sleep00-21swapper/019:09:050
376863699272,6cyclictest4082613-21ntpq00:21:332
3768636992722,4cyclictest4033825-21idleruntime-cro23:35:002
3768636992722,4cyclictest4033825-21idleruntime-cro23:35:002
376863699272,20cyclictest3829400-21ntpq20:06:312
376863699272,20cyclictest3809723-21ntpq19:46:332
3768636992721,4cyclictest4063316-21unixbench_singl00:01:392
376863699271,8cyclictest4005038-21/usr/sbin/munin23:06:242
3768636992714,11cyclictest0-21swapper/219:51:392
3768636992713,12cyclictest0-21swapper/200:06:332
376863699271,20cyclictest3835546-21apt-get20:16:202
3768636992711,14cyclictest0-21swapper/220:01:382
376863699271,10cyclictest3935094-21/usr/sbin/munin21:56:222
3768636992710,14cyclictest3855371-21aten2_r5power_v20:36:212
376863299276,6cyclictest4074237-21cut00:16:271
3768632992721,4cyclictest3989172-21sh22:50:001
3768632992720,5cyclictest4053979-21apt-get23:56:191
3768632992718,2cyclictest161rcu_preempt22:51:331
3768632992716,2cyclictest161rcu_preempt21:40:321
3768632992715,7cyclictest0-21swapper/122:56:311
3768632992715,7cyclictest0-21swapper/100:01:341
3768632992715,4cyclictest161rcu_preempt21:46:281
3768632992714,7cyclictest0-21swapper/123:41:331
3768632992714,7cyclictest0-21swapper/122:01:171
3768632992714,7cyclictest0-21swapper/119:16:311
3768632992714,4cyclictest0-21swapper/119:36:321
3768632992714,2cyclictest161rcu_preempt23:16:341
3768632992713,4cyclictest0-21swapper/123:46:311
3768632992713,4cyclictest0-21swapper/119:56:331
3768632992712,2cyclictest0-21swapper/123:31:371
3768632992712,2cyclictest0-21swapper/123:31:371
376862999271,25cyclictest3894775-21kworker/0:1+events21:28:130
376863699266,18cyclictest3867539-21tr20:46:282
376863699263,13cyclictest0-21swapper/200:16:232
376863699262,4cyclictest3998411-21ntpq22:56:302
3768636992622,2cyclictest3953984-21tail22:11:392
376863699261,9cyclictest3925756-21ps21:46:262
376863699261,8cyclictest4014685-21fschecks_count23:16:232
376863699261,6cyclictest3804738-21ntpq19:41:322
376863699261,5cyclictest4087495-21ntpq00:26:312
376863699261,4cyclictest3944916-21/usr/sbin/munin22:06:252
376863699261,4cyclictest3929961-21/usr/sbin/munin21:51:192
3768636992614,10cyclictest0-21swapper/219:25:002
3768636992614,10cyclictest0-21swapper/219:25:002
376863699261,3cyclictest3963731-21ntpq22:21:352
3768636992610,14cyclictest0-21swapper/221:16:312
376863699260,13cyclictest3900821-21grep21:21:282
3768632992622,2cyclictest0-21swapper/120:56:241
3768632992619,2cyclictest161rcu_preempt22:21:331
3768632992617,2cyclictest161rcu_preempt23:25:061
3768632992616,4cyclictest0-21swapper/123:21:201
3768632992615,2cyclictest161rcu_preempt00:26:381
3768632992614,7cyclictest0-21swapper/122:06:341
3768632992614,7cyclictest0-21swapper/121:51:271
3768632992613,7cyclictest0-21swapper/123:51:351
3768632992613,7cyclictest0-21swapper/119:51:331
3768632992613,4cyclictest0-21swapper/120:16:241
3768632992613,3cyclictest0-21swapper/121:11:231
3768632992613,2cyclictest161rcu_preempt23:36:341
376863699255,6cyclictest4037065-21tr23:36:292
376863699254,4cyclictest3977580-21tr22:36:292
376863699251,8cyclictest3880566-21/usr/sbin/munin21:01:252
376863699251,8cyclictest3880566-21/usr/sbin/munin21:01:252
376863699251,6cyclictest3793515-21tr19:31:292
376863699251,3cyclictest4088916-21/usr/sbin/munin00:31:222
376863699251,10cyclictest3990482-21ps22:51:242
376863699241,6cyclictest3834292-21needreboot20:11:302
376863699241,2cyclictest3979692-21/usr/sbin/munin22:41:212
376863699240,7cyclictest4003848-21grep23:01:392
376863699233,6cyclictest3768635-21cut19:10:002
376863699233,6cyclictest3768635-21cut19:10:002
376863699230,7cyclictest3956415-21tr22:16:272
382036522011,5sleep30-21swapper/320:00:003
37682532193,9sleep30-21swapper/319:07:163
37682532193,9sleep30-21swapper/319:07:163
40776032164,9sleep30-21swapper/300:16:363
40499282132,7sleep30-21swapper/323:51:283
40292462132,7sleep30-21swapper/323:31:213
40292462132,7sleep30-21swapper/323:31:213
39884122132,7sleep30-21swapper/322:46:353
39733382132,7sleep30-21swapper/322:31:303
39543272132,7sleep30-21swapper/322:15:013
38309892132,7sleep30-21swapper/320:11:253
38212502132,7sleep30-21swapper/320:01:253
40813022122,6sleep30-21swapper/300:21:303
40437832122,6sleep30-21swapper/323:45:023
40097462122,6sleep30-21swapper/323:11:233
40039882122,6sleep30-21swapper/323:02:283
39930292122,6sleep30-21swapper/322:51:273
39783932122,6sleep30-21swapper/322:36:323
39662742122,6sleep30-21swapper/322:26:283
39444662122,5sleep30-21swapper/322:05:013
39438702122,6sleep30-21swapper/322:01:363
39205032122,6sleep30-21swapper/321:41:263
39187562122,6sleep30-21swapper/321:36:363
39125042124,5sleep30-21swapper/321:31:313
39087712123,5sleep30-21swapper/321:26:333
38940092122,6sleep30-21swapper/321:11:363
38867712122,6sleep30-21swapper/321:06:303
38737492122,6sleep30-21swapper/320:51:303
38612152122,6sleep30-21swapper/320:41:303
38459082122,5sleep30-21swapper/320:26:253
38445032122,6sleep30-21swapper/320:21:403
38352792122,5sleep30-21swapper/320:15:023
38015452122,5sleep30-21swapper/319:41:273
37960422122,6sleep30-21swapper/319:36:213
37910542122,6sleep30-21swapper/319:31:203
37862042121,7sleep30-21swapper/319:26:223
37818392122,6sleep30-21swapper/319:21:283
37818392122,6sleep30-21swapper/319:21:283
37798172122,6sleep30-21swapper/319:16:393
40934452112,5sleep30-21swapper/300:35:023
40875882112,5sleep30-21swapper/300:26:353
40688252112,5sleep30-21swapper/300:10:263
40643652112,5sleep30-21swapper/300:06:253
40590032112,5sleep30-21swapper/300:01:193
40539832112,5sleep30-21swapper/323:56:213
40430392112,5sleep30-21swapper/323:41:373
40382352112,5sleep30-21swapper/323:36:393
40243092112,5sleep30-21swapper/323:26:203
40192222112,5sleep30-21swapper/323:20:023
40146862112,6sleep30-21swapper/323:16:243
39943742112,5sleep30-21swapper/322:55:023
39800372112,5sleep30-21swapper/322:41:253
39597772112,5sleep30-21swapper/322:20:023
39347462112,5sleep30-21swapper/321:55:023
39344632112,5sleep30-21swapper/321:53:543
39252902112,5sleep30-21swapper/321:46:233
39001112112,6sleep30-21swapper/321:21:233
38951072112,6sleep30-21swapper/321:16:223
38800872112,5sleep30-21swapper/321:01:213
38800872112,5sleep30-21swapper/321:01:213
38756142110,7sleep30-21swapper/320:56:273
38652422112,6sleep30-21swapper/320:46:193
38551942112,5sleep30-21swapper/320:35:283
38508442112,5sleep30-21swapper/320:31:273
38265042112,6sleep30-21swapper/320:06:273
38105362112,5sleep30-21swapper/319:50:023
38058462112,5sleep30-21swapper/319:45:363
37701782112,5sleep30-21swapper/319:11:293
37701782112,5sleep30-21swapper/319:11:293
40045112103,5sleep30-21swapper/323:06:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional