You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 08:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Dec 19, 2025 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
323862238164,24sleep30-21swapper/319:06:173
323352202165,25sleep20-21swapper/219:05:362
325202200163,24sleep10-21swapper/119:08:001
325092200163,25sleep00-21swapper/019:07:510
606321310,4sleep335299cyclictest21:45:253
606321310,4sleep335299cyclictest21:45:253
347998784,2cyclictest0-21swapper/220:59:442
119312660,3sleep011891-21gltestperf22:00:150
294962560,3sleep00-21swapper/021:25:230
151532530,3sleep20-21swapper/219:40:152
108562460,1sleep20-21swapper/223:10:242
143222440,8sleep123-21ksoftirqd/123:20:141
352993415,8cyclictest0-21swapper/321:43:043
35299331,21cyclictest0-21swapper/323:37:503
35299331,13cyclictest0-21swapper/319:45:343
33699329,11cyclictest27095-21perf23:50:000
33699327,12cyclictest18297-21munin-run23:30:010
352993119,3cyclictest0-21swapper/319:14:203
352993115,11cyclictest0-21swapper/322:04:123
352993114,3cyclictest0-21swapper/320:44:003
34299313,17cyclictest27344-40dnf20:08:571
115082310,3sleep111511-21timerwakeupswit19:30:231
352993018,3cyclictest0-21swapper/323:42:083
352993013,11cyclictest0-21swapper/300:12:563
352993013,11cyclictest0-21swapper/300:12:563
35299301,20cyclictest0-21swapper/321:30:403
35299301,10cyclictest0-21swapper/319:29:103
33699301,14cyclictest19881-21munin-run22:20:010
352992913,4cyclictest0-21swapper/323:18:343
34299290,19cyclictest0-21swapper/121:09:251
35299288,3cyclictest0-21swapper/321:11:563
352992817,3cyclictest0-21swapper/322:35:103
352992816,3cyclictest0-21swapper/320:59:223
352992815,9cyclictest0-21swapper/323:57:423
352992815,3cyclictest0-21swapper/321:35:203
352992812,11cyclictest0-21swapper/319:35:523
35299281,12cyclictest0-21swapper/300:35:003
35299280,20cyclictest0-21swapper/322:32:083
33699281,13cyclictest29292-21perf23:55:010
35299279,3cyclictest0-21swapper/322:45:003
35299278,9cyclictest0-21swapper/300:38:263
35299275,12cyclictest0-21swapper/322:48:283
352992717,4cyclictest0-21swapper/320:22:543
352992714,3cyclictest0-21swapper/321:55:103
352992712,10cyclictest0-21swapper/319:55:503
352992711,7cyclictest0-21swapper/321:20:403
35299271,13cyclictest0-21swapper/322:13:343
35299270,12cyclictest0-21swapper/322:29:083
35299270,10cyclictest0-21swapper/319:30:223
35299269,3cyclictest0-21swapper/320:19:343
35299268,3cyclictest0-21swapper/323:06:003
35299268,3cyclictest0-21swapper/323:02:503
35299268,3cyclictest0-21swapper/322:50:083
35299268,2cyclictest0-21swapper/322:20:183
352992616,3cyclictest0-21swapper/320:36:203
352992614,3cyclictest0-21swapper/323:12:083
352992614,3cyclictest0-21swapper/300:28:403
352992613,8cyclictest0-21swapper/319:43:223
35299261,20cyclictest0-21swapper/300:01:023
35299261,16cyclictest0-21swapper/323:27:543
352992611,3cyclictest0-21swapper/322:19:403
35299261,12cyclictest21468-21diskmemload22:06:283
35299260,12cyclictest0-21swapper/300:22:243
34799261,6cyclictest0-21swapper/221:27:122
34299263,3cyclictest17470-21sshd22:12:261
33699263,12cyclictest10157-21awk00:25:010
35299259,3cyclictest24629-21if_enp1s021:15:163
35299259,3cyclictest0-21swapper/319:54:283
35299258,7cyclictest0-21swapper/321:27:043
35299258,3cyclictest0-21swapper/319:23:223
35299258,3cyclictest0-21swapper/319:23:223
352992521,3cyclictest0-21swapper/320:28:563
352992511,3cyclictest0-21swapper/320:02:543
35299251,11cyclictest0-21swapper/300:07:563
352992510,9cyclictest0-21swapper/320:05:563
34299256,8cyclictest0-21swapper/121:33:201
33699256,16cyclictest21472-21munin-run21:10:010
33699251,12cyclictest20857-21awk19:55:010
35299249,3cyclictest0-21swapper/320:10:023
35299248,7cyclictest0-21swapper/323:22:363
35299241,8cyclictest26027-21memory23:45:183
352992410,3cyclictest0-21swapper/320:34:143
34799241,7cyclictest0-21swapper/200:05:382
34799241,6cyclictest0-21swapper/221:47:582
34799241,6cyclictest0-21swapper/221:47:582
34299246,8cyclictest0-21swapper/123:16:491
34299242,19cyclictest11086-21munin-run22:00:011
34299242,15cyclictest87250irq/129-enp1s0-00:10:581
34299242,15cyclictest87250irq/129-enp1s0-00:10:581
342992418,3cyclictest7469-21perl21:50:151
33699248,14cyclictest14547-21awk00:35:010
33699247,15cyclictest4507-21awk21:45:010
33699247,14cyclictest19283-21idleruntime-cro21:05:010
33699241,11cyclictest24277-21munin-run22:30:000
35299239,3cyclictest0-21swapper/320:52:563
352992315,3cyclictest0-21swapper/300:19:023
35299231,12cyclictest0-21swapper/321:51:363
352992310,3cyclictest0-21swapper/322:57:083
352992310,3cyclictest0-21swapper/322:57:083
35299230,12cyclictest0-21swapper/323:51:143
34799237,5cyclictest904-21sssd_nss20:30:272
34799232,4cyclictest0-21swapper/221:10:182
34299235,8cyclictest0-21swapper/121:13:431
34299231,10cyclictest8151-21cat00:20:011
34299231,10cyclictest6197-21cat20:35:011
33699238,3cyclictest24127-21latency20:00:180
33699234,16cyclictest31846-21awk20:20:000
33699233,13cyclictest0-21swapper/023:12:190
33699232,2cyclictest0-21swapper/000:19:230
33699231,9cyclictest11517-21grep00:25:220
336992316,4cyclictest31094-21grep21:30:160
35299228,3cyclictest0-21swapper/321:05:483
35299226,3cyclictest0-21swapper/323:30:583
352992210,3cyclictest0-21swapper/320:46:323
34799221,7cyclictest0-21swapper/222:31:552
34799221,6cyclictest0-21swapper/222:15:342
34299225,4cyclictest0-21swapper/122:58:591
34299225,4cyclictest0-21swapper/122:58:591
34299224,6cyclictest11140-40dnf23:10:291
34299221,3cyclictest416-21irqstats20:20:191
34299221,3cyclictest19332-21irqstats23:30:181
34299221,19cyclictest0-21swapper/122:30:221
34299221,18cyclictest19033-21expr19:50:131
34299221,14cyclictest87250irq/129-enp1s0-22:01:001
34299221,12cyclictest0-21swapper/122:08:171
34299221,11cyclictest21643-21cat23:35:211
34299220,20cyclictest0-21swapper/123:46:481
33699223,14cyclictest0-21swapper/021:47:090
33699223,14cyclictest0-21swapper/021:47:090
33699223,12cyclictest0-21swapper/021:51:160
33699222,15cyclictest0-21swapper/022:21:280
33699221,6cyclictest0-21swapper/021:38:380
33699221,5cyclictest24801-21memory21:15:180
35299211,12cyclictest0-21swapper/321:00:183
34799211,6cyclictest0-21swapper/223:56:472
34799211,6cyclictest0-21swapper/223:30:202
34799211,6cyclictest0-21swapper/200:21:182
34799211,5cyclictest0-21swapper/223:08:582
34799211,5cyclictest0-21swapper/222:06:462
34299213,3cyclictest87150irq/128-enp1s0-21:40:231
34299212,6cyclictest17135-21irqstats23:25:181
342992117,3cyclictest0-21swapper/122:24:451
34299211,6cyclictest0-21swapper/121:28:091
34299211,5cyclictest3008-21latency_hist22:55:011
34299211,3cyclictest31478-21munin-run00:00:011
34299211,17cyclictest30876-21sleep22:45:011
34299210,13cyclictest0-21swapper/121:15:041
33699215,5cyclictest986-1kworker/u9:122:30:540
33699213,11cyclictest0-21swapper/023:36:360
33699211,7cyclictest0-21swapper/022:48:310
33699211,5cyclictest22566-21latency21:10:190
33699211,4cyclictest804-21lldpd19:47:410
33699211,3cyclictest0-21swapper/000:01:080
35299208,3cyclictest0-21swapper/319:17:303
34799202,9cyclictest2026550kworker/u8:200:18:042
34799202,7cyclictest628-21munin-run22:50:012
34799201,9cyclictest302-21ntp_states00:00:252
34799201,5cyclictest0-21swapper/223:02:122
34799201,4cyclictest0-21swapper/222:40:412
347992014,3cyclictest29632-21idleruntime-cro20:15:002
34799201,13cyclictest0-21swapper/221:44:082
34799201,12cyclictest0-21swapper/223:39:182
34299209,3cyclictest3204-21sed20:25:251
34299208,3cyclictest4768-21smart_sda19:15:231
34299203,7cyclictest7893-21/usr/sbin/munin19:25:241
34299202,6cyclictest7672-21munin-run19:25:011
34299202,6cyclictest7672-21munin-run19:25:011
34299202,5cyclictest14256-21munin-run19:40:011
34299202,3cyclictest1374-21ntp_states21:35:221
34299202,15cyclictest14863-21munin-run20:55:011
342992016,3cyclictest0-21swapper/120:56:051
34299201,17cyclictest0-21swapper/120:02:231
34299201,17cyclictest0-21swapper/100:35:191
34299201,13cyclictest0-21swapper/122:47:591
34299201,10cyclictest28668-21/usr/sbin/munin22:40:011
34299201,10cyclictest27327-21egrep23:50:121
34299201,10cyclictest17550-21latency19:45:181
34299200,9cyclictest0-21swapper/120:45:111
34299200,13cyclictest87250irq/129-enp1s0-23:40:201
33699203,13cyclictest8665-21tr20:40:130
33699203,10cyclictest0-21swapper/022:44:320
33699201,3cyclictest0-21swapper/021:57:410
33699201,2cyclictest0-21swapper/022:40:010
336992011,8cyclictest130-21kworker/u8:320:13:520
33699201,11cyclictest0-21swapper/023:43:250
33699201,11cyclictest0-21swapper/021:22:570
33699201,10cyclictest0-21swapper/022:55:120
33699201,10cyclictest0-21swapper/022:55:120
34799191,7cyclictest0-21swapper/221:30:192
34799191,5cyclictest986-1kworker/u9:121:51:242
34799191,4cyclictest0-21swapper/221:22:192
34799191,15cyclictest10210-21cut19:30:112
34299196,3cyclictest0-21swapper/122:25:121
34299195,4cyclictest0-21swapper/100:02:121
34299194,5cyclictest0-21swapper/123:07:581
34299193,14cyclictest12712-21tr00:30:131
34299192,16cyclictest0-21swapper/121:22:021
34299191,6cyclictest15630-21pmu-power19:40:221
34299191,6cyclictest0-21swapper/100:25:091
342991915,3cyclictest0-21swapper/122:15:381
342991915,3cyclictest0-21swapper/120:16:101
34299191,3cyclictest0-21swapper/121:47:411
34299191,3cyclictest0-21swapper/121:47:411
34299191,17cyclictest0-21swapper/120:36:291
34299191,17cyclictest0-21swapper/120:11:261
34299191,17cyclictest0-21swapper/119:55:521
34299191,17cyclictest0-21swapper/100:06:341
34299191,11cyclictest10126-21dump-pmu-power00:24:561
34299191,11cyclictest0-21swapper/120:40:241
34299190,9cyclictest0-21swapper/123:04:481
33699195,7cyclictest0-21swapper/022:52:080
33699194,4cyclictest0-21swapper/023:00:210
33699191,3cyclictest0-21swapper/020:05:250
33699190,5cyclictest0-21swapper/023:22:360
33699190,3cyclictest0-21swapper/023:08:540
34799182,3cyclictest23088-21cstates23:40:132
34799181,7cyclictest0-21swapper/221:55:412
34799181,4cyclictest0-21swapper/223:46:442
34799181,3cyclictest0-21swapper/221:39:412
347991813,3cyclictest17429-21tr21:00:122
347991813,3cyclictest12894-21/usr/sbin/munin20:50:272
34799181,12cyclictest0-21swapper/222:12:302
34799181,11cyclictest0-21swapper/222:27:402
34799181,10cyclictest0-21swapper/223:15:172
34799181,10cyclictest0-21swapper/221:17:442
342991815,2cyclictest0-21swapper/119:14:101
34299181,15cyclictest0-21swapper/121:00:051
33699188,3cyclictest801-21snmpd19:36:310
33699185,8cyclictest0-21swapper/023:31:150
33699182,9cyclictest0-21swapper/000:05:260
33699181,6cyclictest0-21swapper/022:12:140
33699181,6cyclictest0-21swapper/020:55:130
33699180,5cyclictest0-21swapper/022:07:450
33699180,4cyclictest0-21swapper/000:37:200
5112170,3sleep20-21swapper/219:10:112
34799176,3cyclictest87250irq/129-enp1s0-22:55:192
34799176,3cyclictest87250irq/129-enp1s0-22:55:192
34799173,3cyclictest86950irq/126-enp1s0-00:25:192
34799172,13cyclictest15420-21grep00:35:162
34799172,13cyclictest11650-21seq20:45:182
34799171,7cyclictest27439-21latency_hist20:10:012
34799171,6cyclictest28483-21/usr/sbin/munin23:50:232
34799171,6cyclictest0-21swapper/223:25:022
34799171,6cyclictest0-21swapper/223:24:522
34799171,6cyclictest0-21swapper/222:50:162
34799171,3cyclictest0-21swapper/220:43:502
347991712,3cyclictest6393-21ls20:35:132
347991712,3cyclictest31310-21sed20:15:262
34799171,15cyclictest0-21swapper/219:37:262
34799171,10cyclictest0-21swapper/222:00:232
33699175,3cyclictest12672-21if_enp1s023:15:170
33699175,3cyclictest0-21swapper/020:29:260
33699173,8cyclictest0-21swapper/023:55:180
33699172,5cyclictest6076-21latency_hist20:35:010
33699171,4cyclictest11758-21ntp_states20:45:210
33699171,3cyclictest0-21swapper/020:35:220
336991712,3cyclictest4764-21sensors_temp19:15:230
33699171,14cyclictest13507-21grep20:50:180
336991711,3cyclictest4638-21sort00:10:200
336991711,3cyclictest4638-21sort00:10:200
33699171,10cyclictest0-21swapper/019:10:120
227382170,2sleep20-21swapper/219:56:222
199422171,2sleep20-21swapper/219:50:242
76162160,2sleep00-21swapper/019:24:290
76162160,2sleep00-21swapper/019:24:290
34799162,5cyclictest0-21swapper/222:20:252
34799161,6cyclictest14043-21unixbench-2d00:30:262
347991612,3cyclictest0-21swapper/222:35:392
347991612,3cyclictest0-21swapper/220:02:082
347991612,3cyclictest0-21swapper/200:12:312
347991612,3cyclictest0-21swapper/200:12:312
34799161,14cyclictest0-21swapper/220:20:022
347991611,3cyclictest2974-21munin-node19:15:022
347991611,3cyclictest17010-21df19:45:132
347991610,3cyclictest20462-21cat21:05:202
34799160,15cyclictest0-21swapper/219:26:122
34799160,14cyclictest0-21swapper/220:25:302
33699164,3cyclictest22213-21pmu-power19:55:230
33699161,2cyclictest0-21swapper/019:25:110
33699161,12cyclictest32061-21/usr/sbin/munin20:20:180
336991610,3cyclictest14814-21df19:40:130
34799151,2cyclictest0-21swapper/219:20:082
34799151,2cyclictest0-21swapper/219:20:082
33699159,3cyclictest10303-21cat19:30:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional