You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 10:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Thu Jun 04, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
168352236163,24sleep10-21swapper/119:07:321
169082230160,56sleep20-21swapper/219:08:292
167122201164,24sleep30-21swapper/319:05:573
168342200162,26sleep00-21swapper/019:07:310
17125997976,2cyclictest0-21swapper/200:34:452
17125996360,2cyclictest0-21swapper/220:13:262
48812520,1sleep20-21swapper/200:20:162
131082490,2sleep10-21swapper/122:10:241
17125994840,2cyclictest0-21swapper/221:44:312
206252420,4sleep121-21rcuc/123:40:261
1713299360,25cyclictest0-21swapper/322:15:193
17125993431,2cyclictest0-21swapper/220:02:372
1712099331,23cyclictest8741-21hddtemp_smartct23:15:161
1713299321,22cyclictest0-21swapper/320:57:273
1712099321,22cyclictest27161-21sed22:45:021
1713299301,9cyclictest0-21swapper/321:11:313
1713299300,23cyclictest0-21swapper/323:19:113
1713299300,20cyclictest0-21swapper/321:40:433
1713299300,14cyclictest0-21swapper/319:59:033
1712099301,23cyclictest0-21swapper/123:50:011
1711599301,14cyclictest28415-21munin-run19:45:010
1713299291,11cyclictest0-21swapper/323:07:473
1713299290,23cyclictest0-21swapper/321:06:113
1713299290,23cyclictest0-21swapper/321:06:113
1712599290,16cyclictest0-21swapper/220:41:352
1712099291,26cyclictest0-21swapper/119:40:141
1713299281,9cyclictest0-21swapper/320:39:553
1713299281,19cyclictest0-21swapper/319:27:473
1713299281,10cyclictest0-21swapper/321:51:173
1713299280,21cyclictest0-21swapper/320:25:433
1713299280,17cyclictest0-21swapper/323:32:073
1713299280,17cyclictest0-21swapper/323:32:073
1713299280,17cyclictest0-21swapper/321:35:273
1713299280,13cyclictest0-21swapper/320:07:593
1713299280,11cyclictest0-21swapper/300:30:153
1712099282,24cyclictest0-21swapper/119:53:131
1712099282,22cyclictest13549-21munin-run20:50:011
1712099281,25cyclictest0-21swapper/120:25:591
1712099280,20cyclictest0-21swapper/100:23:211
1711599288,17cyclictest26214-21munin-run19:40:010
1713299271,9cyclictest0-21swapper/322:41:273
1713299271,11cyclictest0-21swapper/320:48:393
1713299271,11cyclictest0-21swapper/319:33:393
1713299271,10cyclictest0-21swapper/322:22:593
1713299271,10cyclictest0-21swapper/322:22:593
1713299270,13cyclictest0-21swapper/321:23:033
1713299270,11cyclictest0-21swapper/322:01:543
1712099272,12cyclictest16021-21munin-run22:20:011
1712099272,11cyclictest20591-21munin-run19:20:011
1712099271,24cyclictest0-21swapper/123:59:051
1712099271,24cyclictest0-21swapper/120:37:101
1712099271,24cyclictest0-21swapper/119:32:361
1712099271,17cyclictest0-21swapper/121:03:311
17120992711,3cyclictest256852smart_sda21:25:241
1711599275,10cyclictest15759-21perf20:55:000
1711599273,12cyclictest10737-21munin-run20:40:020
17115992710,14cyclictest7860-21perf23:15:010
1711599270,13cyclictest7237-21perf22:00:010
157632270,2sleep00-21swapper/023:30:210
157632270,2sleep00-21swapper/023:30:210
1713299261,9cyclictest0-21swapper/320:44:513
1713299261,13cyclictest0-21swapper/300:38:113
1713299261,11cyclictest19186-21diskmemload21:29:273
1713299261,11cyclictest0-21swapper/300:29:513
1713299261,11cyclictest0-21swapper/300:18:433
1713299261,10cyclictest0-21swapper/323:28:353
1713299260,11cyclictest0-21swapper/323:43:433
1713299260,10cyclictest0-21swapper/322:07:543
1713299260,10cyclictest0-21swapper/321:03:433
1712099262,20cyclictest23404-21munin-run19:30:001
1712099261,23cyclictest0-21swapper/123:36:501
1712099261,15cyclictest0-21swapper/122:36:301
1712099260,24cyclictest0-21swapper/120:33:161
1712099260,24cyclictest0-21swapper/120:02:081
1712099260,24cyclictest0-21swapper/119:46:361
1712099260,24cyclictest0-21swapper/100:06:231
1712099260,24cyclictest0-21swapper/100:03:191
256872250,2sleep0909-21sssd_nss21:25:240
1713299251,2cyclictest87550irq/128-enp1s0-23:00:033
1713299251,16cyclictest0-21swapper/322:25:433
1713299251,10cyclictest0-21swapper/320:02:213
1713299250,9cyclictest0-21swapper/323:52:233
1713299250,9cyclictest0-21swapper/320:50:313
1712099252,15cyclictest0-21swapper/122:08:151
1712099251,3cyclictest20430-21perf22:30:011
1712099251,22cyclictest0-21swapper/123:30:261
1712099251,22cyclictest0-21swapper/123:30:261
1712099251,22cyclictest0-21swapper/100:14:091
1712099251,20cyclictest3120-21/usr/sbin/munin21:50:131
1712099251,15cyclictest0-21swapper/122:57:131
1712099251,15cyclictest0-21swapper/122:49:161
1712099251,15cyclictest0-21swapper/121:55:351
1712099251,12cyclictest12156-21chrt23:24:071
1712099251,11cyclictest968-21kworker/u8:023:27:591
1712099250,23cyclictest0-21swapper/120:41:251
1712099250,23cyclictest0-21swapper/120:21:111
1712099250,23cyclictest0-21swapper/100:33:401
1713299241,8cyclictest0-21swapper/319:48:313
1713299241,12cyclictest31114-21runrttasks19:53:433
1713299241,12cyclictest0-21swapper/319:15:153
1713299241,11cyclictest0-21swapper/319:14:383
1713299240,17cyclictest0-21swapper/321:16:393
1713299240,17cyclictest0-21swapper/320:18:463
1713299240,12cyclictest0-21swapper/323:46:553
1713299240,11cyclictest0-21swapper/323:23:433
1713299240,11cyclictest0-21swapper/323:14:313
1713299240,11cyclictest0-21swapper/322:12:463
1713299240,10cyclictest0-21swapper/321:57:473
1712599241,3cyclictest0-21swapper/222:38:312
1712099247,3cyclictest0-21swapper/123:10:121
17120992413,3cyclictest0-21swapper/122:34:581
17120992413,3cyclictest0-21swapper/121:47:421
17120992413,3cyclictest0-21swapper/121:31:571
1712099241,20cyclictest19759-21diskstats21:10:101
1712099241,14cyclictest0-21swapper/121:18:441
1712099241,11cyclictest3675-21/usr/sbin/munin20:10:181
1712099241,11cyclictest3380-21smart_sda00:15:251
1712099241,11cyclictest24438-21needreboot23:50:191
1712099241,11cyclictest19332-21munin-node19:15:021
1712099240,15cyclictest0-21swapper/122:05:011
1712099240,14cyclictest0-21swapper/122:20:131
1712099240,14cyclictest0-21swapper/122:20:131
1711599241,12cyclictest32064-21awk00:10:020
1713299239,3cyclictest18833-21latency_hist23:40:013
1713299236,3cyclictest0-21swapper/320:31:213
17132992312,6cyclictest0-21swapper/323:55:433
1713299231,11cyclictest0-21swapper/319:21:313
1713299230,12cyclictest0-21swapper/321:48:473
1713299230,10cyclictest0-21swapper/322:37:513
17125992317,3cyclictest7222-21turbostat22:00:012
17125992313,3cyclictest0-21swapper/223:43:492
17120992312,3cyclictest0-21swapper/120:52:181
1712099231,11cyclictest1655-21cstates23:00:131
1712099231,10cyclictest12630-21taskset00:37:051
181252220,4sleep230-21rcuc/219:10:182
1713299228,3cyclictest0-21swapper/321:34:313
1713299221,9cyclictest0-21swapper/322:52:113
1713299221,8cyclictest0-21swapper/322:57:033
1713299221,8cyclictest0-21swapper/320:20:473
1713299221,11cyclictest0-21swapper/319:40:433
17132992210,3cyclictest0-21swapper/322:31:513
1713299220,10cyclictest0-21swapper/322:45:193
1712599221,6cyclictest0-21swapper/222:34:262
1712599221,3cyclictest87350irq/126-enp1s0-22:12:342
1712099223,10cyclictest5931-21kworker/u8:200:29:491
17120992218,3cyclictest0-21swapper/119:55:471
1712099221,3cyclictest28875-21kworker/u8:222:53:401
17120992212,3cyclictest0-21swapper/121:21:141
1712099221,19cyclictest0-21swapper/120:56:181
1712099221,10cyclictest3940-21expr23:05:111
1711599221,3cyclictest0-21swapper/000:11:240
1713299217,3cyclictest0-21swapper/300:23:573
1713299217,3cyclictest0-21swapper/300:07:003
1713299211,7cyclictest0-21swapper/319:39:053
1713299211,3cyclictest0-21swapper/300:11:353
1713299211,11cyclictest0-21swapper/300:02:213
1713299211,10cyclictest0-21swapper/320:11:383
1712599214,11cyclictest0-21swapper/223:29:322
1712599211,13cyclictest0-21swapper/222:45:222
1712599211,11cyclictest27626-21latency_hist00:00:022
1712099212,18cyclictest0-21swapper/119:21:581
1712099211,18cyclictest0-21swapper/120:06:581
17120992110,3cyclictest0-21swapper/121:40:301
17120992110,3cyclictest0-21swapper/121:36:551
1711599218,2cyclictest0-21swapper/023:05:130
1711599215,13cyclictest23596-21cat19:30:010
1712599203,14cyclictest8659-21grep23:15:152
1712599201,8cyclictest0-21swapper/222:16:042
1712599201,7cyclictest4737-21runrttasks23:05:192
1712599201,15cyclictest20878-21cpuspeed_turbos19:20:082
1711599201,5cyclictest0-21swapper/000:26:000
1711599201,3cyclictest0-21swapper/019:45:380
17115992013,4cyclictest10397-21grep23:20:110
1711599201,11cyclictest27852-21/usr/sbin/munin00:00:240
1711599201,10cyclictest24970-21grep23:50:250
233632191,2sleep20-21swapper/219:29:412
1712599197,2cyclictest30-21rcuc/222:55:132
1712599196,3cyclictest87350irq/126-enp1s0-19:55:012
1712599194,9cyclictest0-21swapper/221:10:472
1712599194,4cyclictest19186-21diskmemload00:37:282
1712599193,9cyclictest0-21swapper/221:35:052
1712599193,4cyclictest755-21rngd23:50:222
1712599193,4cyclictest31132-21ntp_states00:05:212
1712599192,8cyclictest12050irq/123-ahci[0021:55:012
1712599191,6cyclictest15738-21latency_hist20:55:012
1712599191,6cyclictest0-21swapper/222:05:232
1712599191,5cyclictest2460-21needreboot23:00:192
1712599191,10cyclictest0-21swapper/223:35:332
1712599191,10cyclictest0-21swapper/221:29:212
1712599190,9cyclictest0-21swapper/219:40:202
1712099191,16cyclictest0-21swapper/120:15:571
1712099190,2cyclictest0-21swapper/119:35:131
1711599194,2cyclictest87550irq/128-enp1s0-21:18:370
1711599191,5cyclictest24655-1kworker/u9:022:23:080
1711599191,5cyclictest24655-1kworker/u9:022:23:080
17115991914,3cyclictest23777-21sed21:20:230
1711599191,12cyclictest0-21swapper/022:35:200
1711599191,10cyclictest32062-21/usr/sbin/munin20:00:110
1711599191,10cyclictest28820-21/usr/sbin/munin21:35:170
1711599191,10cyclictest0-21swapper/022:07:090
129062191,2sleep00-21swapper/020:44:520
26542180,3sleep00-21swapper/021:47:570
258432181,2sleep20-21swapper/219:35:272
1712599188,2cyclictest23999-21fschecks_count19:30:112
1712599184,3cyclictest19186-21diskmemload23:14:432
1712599184,3cyclictest19186-21diskmemload21:20:122
1712599183,3cyclictest87350irq/126-enp1s0-00:15:272
1712599181,9cyclictest2860-1kworker/2:2H20:29:562
1712599181,9cyclictest0-21swapper/220:32:522
1712599181,7cyclictest0-21swapper/200:03:032
1712599181,6cyclictest0-21swapper/222:28:092
1712599181,5cyclictest394-21gltestperf00:10:162
1712599181,5cyclictest26580-21unixbench_singl22:40:252
1712599181,4cyclictest22666-21grep23:45:242
17125991812,4cyclictest18567-21crond21:05:012
1712599181,15cyclictest0-21swapper/222:00:102
1712599181,11cyclictest0-21swapper/222:24:222
1712599181,11cyclictest0-21swapper/222:24:222
1711599188,3cyclictest8068-21/usr/sbin/munin23:15:170
1711599181,7cyclictest30584-21head22:50:220
1711599181,7cyclictest26363-21sort23:55:150
1711599181,5cyclictest0-21swapper/023:04:410
17115991813,3cyclictest9968-21smartctl00:30:240
17115991813,3cyclictest8128-21smartctl22:00:170
17115991813,3cyclictest27917-21smartctl22:45:160
1711599181,11cyclictest0-21swapper/022:15:220
1711599181,11cyclictest0-21swapper/021:11:000
1711599180,15cyclictest17284-21grep23:35:150
141592180,3sleep00-21swapper/023:26:410
181962171,2sleep00-21swapper/021:00:290
1712599172,3cyclictest31888-21cat20:00:002
1712599171,9cyclictest0-21swapper/220:20:222
1712599171,9cyclictest0-21swapper/219:15:342
1712599171,8cyclictest87650irq/129-enp1s0-20:19:362
1712599171,6cyclictest0-21swapper/223:34:372
1712599171,6cyclictest0-21swapper/223:34:372
1712599171,4cyclictest2825-21latency_hist21:50:022
1712599171,4cyclictest26963-21df21:30:132
1712599171,4cyclictest0-21swapper/221:20:012
17125991712,3cyclictest10898-21tail23:20:152
1712599171,15cyclictest0-21swapper/219:48:592
1712599171,10cyclictest0-21swapper/221:06:532
1712599171,10cyclictest0-21swapper/221:06:532
1712599171,10cyclictest0-21swapper/200:28:212
1712099171,2cyclictest0-21swapper/121:05:231
1712099171,2cyclictest0-21swapper/121:05:221
1711599177,9cyclictest31849-21awk20:00:010
1711599176,8cyclictest4972-21gltestperf00:20:160
1711599176,4cyclictest1881-21kworker/u8:000:15:140
1711599175,4cyclictest12357-21timerwakeupswit00:35:230
1711599174,12cyclictest0-21swapper/020:35:010
1711599171,7cyclictest21262-21/usr/sbin/munin23:45:230
1711599171,7cyclictest20652-21/usr/sbin/munin22:30:200
1711599171,6cyclictest0-21swapper/022:57:120
1711599171,6cyclictest0-21swapper/021:32:240
17115991712,3cyclictest25137-21cpuspeed_turbos22:40:110
1711599171,15cyclictest0-21swapper/021:54:160
1711599171,14cyclictest0-21swapper/020:10:360
1711599170,8cyclictest0-21swapper/021:40:180
104342170,2sleep20-21swapper/220:36:232
1712599161,9cyclictest0-21swapper/220:58:592
1712599161,6cyclictest0-21swapper/220:47:092
1712599161,5cyclictest0-21swapper/222:52:002
1712599161,2cyclictest0-21swapper/220:06:262
1711599167,2cyclictest29248-21/usr/sbin/munin19:50:150
1711599165,3cyclictest87550irq/128-enp1s0-22:25:240
1711599161,6cyclictest23974-21diskstats19:30:100
1711599161,6cyclictest19175-21grep23:40:120
1711599161,5cyclictest0-21swapper/022:11:080
1711599161,4cyclictest808-21snmpd19:18:470
17115991611,3cyclictest763-21dbus-daemon19:15:010
17115991611,3cyclictest11-21rcuc/021:06:420
17115991611,3cyclictest11-21rcuc/021:06:420
1711599160,14cyclictest0-21swapper/020:15:370
1711599160,10cyclictest0-21swapper/019:20:180
159572161,2sleep00-21swapper/020:55:030
1711599155,3cyclictest81ktimersoftd/020:30:010
1711599152,4cyclictest5353-21egrep20:20:080
1711599151,7cyclictest808-21snmpd20:07:360
17115991511,3cyclictest0-21swapper/020:46:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional