You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 19:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Jun 14, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
241172235164,22sleep20-21swapper/207:09:152
241172235164,22sleep20-21swapper/207:09:152
241652217146,58sleep10-21swapper/107:09:511
241652217146,58sleep10-21swapper/107:09:501
238682204165,26sleep00-21swapper/007:06:000
238682204165,26sleep00-21swapper/007:05:590
238662201165,24sleep30-21swapper/307:05:583
238662201165,24sleep30-21swapper/307:05:583
216221390,3sleep22427399cyclictest11:10:172
24273998177,3cyclictest0-21swapper/209:31:122
24273998177,3cyclictest0-21swapper/209:31:122
23992540,3sleep011-21rcuc/008:45:120
197402450,1sleep10-21swapper/110:35:261
273322420,2sleep00-21swapper/009:40:190
24278993716,5cyclictest30271-21ntp_states11:00:203
24278993711,9cyclictest827-21runrttasks09:38:373
24278993611,8cyclictest790-21in:imjournal11:50:023
309342350,4sleep07-21ksoftirqd/008:35:160
2427899358,20cyclictest14318-21pmu-power09:10:243
24278993512,9cyclictest5118-21kworker/u8:111:25:133
24278993512,14cyclictest1171-21fschecks_time09:55:163
24278993510,18cyclictest0-21swapper/310:45:123
2426899350,24cyclictest0-21swapper/109:10:011
2427899349,14cyclictest0-21swapper/309:40:133
24278993415,15cyclictest5025-40dnf07:35:223
2426899341,3cyclictest0-21swapper/108:39:451
2427899339,10cyclictest3860-21fschecks_count11:15:153
2427899338,13cyclictest27628-21/usr/sbin/munin12:10:123
2427899336,18cyclictest10905-21sed10:15:253
24278993315,10cyclictest0-21swapper/311:09:343
24278993313,11cyclictest23218-21/usr/sbin/munin12:00:243
24278993311,13cyclictest12968-21diskmemload11:32:543
24278993310,9cyclictest11614-21/usr/sbin/munin10:20:153
2427899329,6cyclictest4210-21cpuspeed_turbos12:30:103
24278993215,5cyclictest7790-21unixbench_singl12:35:253
24278993213,6cyclictest790-21in:imjournal11:15:023
24278993211,10cyclictest20836-21ntpq09:25:213
2427899319,12cyclictest16567-21cat11:45:023
24278993115,4cyclictest14883-21ntp_states10:25:233
24278993114,5cyclictest87350irq/126-enp1s0-12:25:243
24278993113,5cyclictest8565-21latency_hist09:00:023
24278993111,8cyclictest17036-21cpu08:05:143
24278993111,11cyclictest0-21swapper/312:20:263
2426899311,9cyclictest0-21swapper/108:43:491
2426899311,21cyclictest0-21swapper/109:34:251
2426899311,21cyclictest0-21swapper/109:34:251
2426899311,19cyclictest0-21swapper/112:13:011
2426899310,23cyclictest0-21swapper/110:48:051
2427899308,13cyclictest20406-21/usr/sbin/munin10:40:163
2427899308,13cyclictest20406-21/usr/sbin/munin10:40:163
24278993021,5cyclictest27704-21diskstats07:15:153
24278993017,5cyclictest12230-21/usr/sbin/munin11:35:123
24278993013,13cyclictest6868-21grep11:20:233
24278993012,7cyclictest15242-21fschecks_time08:00:163
2426899301,18cyclictest0-21swapper/107:18:211
2426899300,11cyclictest0-21swapper/108:04:541
2427899299,5cyclictest12050irq/123-ahci[0008:55:013
2427899298,8cyclictest0-21swapper/309:30:133
2427899298,8cyclictest0-21swapper/309:30:133
2427899298,8cyclictest0-21swapper/308:35:203
24278992922,4cyclictest19525-21/usr/sbin/munin08:10:143
24278992919,6cyclictest16817-21/usr/sbin/munin10:30:153
24278992916,10cyclictest3790-21/usr/sbin/munin08:45:253
24278992916,10cyclictest26310-21sshd07:13:333
24278992913,6cyclictest87350irq/126-enp1s0-11:50:153
24278992912,9cyclictest5842-21irqstats10:05:213
24278992911,14cyclictest31959-21grep09:50:233
24278992910,16cyclictest30398-21taskset09:48:083
2426899291,11cyclictest0-21swapper/107:30:171
24268992910,3cyclictest0-21swapper/108:25:381
2426899290,21cyclictest0-21swapper/112:39:291
2427899288,11cyclictest0-21swapper/307:50:223
24278992821,4cyclictest26495-21ntp_states12:05:213
24278992820,5cyclictest21719-21diskstats08:15:133
24278992820,5cyclictest21719-21diskstats08:15:133
24278992820,5cyclictest19274-21ntp_states10:35:203
24278992819,6cyclictest29230-21/usr/sbin/munin08:30:253
24278992819,6cyclictest25220-21tune2fs10:50:143
24278992819,6cyclictest12904-21cat07:55:133
24278992819,6cyclictest11364-21/usr/sbin/munin09:05:163
24278992819,5cyclictest1937-21diskstats07:30:143
24278992813,7cyclictest16204-21irqstats09:15:183
24278992812,13cyclictest21358-21df11:55:143
2426899281,9cyclictest0-21swapper/107:13:291
2426899281,10cyclictest0-21swapper/108:49:211
2426899280,22cyclictest0-21swapper/109:12:291
2426899280,13cyclictest0-21swapper/110:02:131
2426899280,12cyclictest0-21swapper/110:10:371
2427899277,8cyclictest0-21swapper/309:20:143
2427899276,7cyclictest0-21swapper/308:40:183
24278992720,4cyclictest30173-21diskstats12:15:143
24278992720,4cyclictest26845-21ntp_states08:25:223
24278992718,6cyclictest9464-21hddtemp_smartct09:00:163
24278992718,6cyclictest9246-21ntp_states07:45:213
24278992718,6cyclictest7221-21/usr/sbin/munin10:10:203
24278992718,6cyclictest6331-21diskstats07:40:143
24278992718,6cyclictest3909-21/usr/sbin/munin10:00:223
24278992718,6cyclictest27357-21/usr/sbin/munin10:55:123
2426899271,11cyclictest0-21swapper/109:15:331
2426899271,10cyclictest0-21swapper/112:06:171
2426899271,10cyclictest0-21swapper/109:50:051
2426899271,10cyclictest0-21swapper/107:22:491
2426899270,11cyclictest0-21swapper/110:33:021
2426899270,10cyclictest0-21swapper/108:05:261
2426099278,16cyclictest25548-21munin-run08:25:010
2426099271,13cyclictest1002-21munin-run11:10:010
24278992619,4cyclictest32084-21df07:25:133
24278992618,5cyclictest30223-21hddtemp_smartct07:20:163
24278992618,5cyclictest23919-21/usr/sbin/munin08:20:143
2426899262,11cyclictest6930-21df08:55:111
2426899261,9cyclictest0-21swapper/107:43:331
2426899261,11cyclictest0-21swapper/107:51:281
2426899261,11cyclictest0-21swapper/107:29:371
2426899261,10cyclictest0-21swapper/111:53:291
2426899261,10cyclictest0-21swapper/111:25:411
2426899260,9cyclictest0-21swapper/111:22:571
2426899260,2cyclictest0-21swapper/109:58:491
2426899260,10cyclictest0-21swapper/112:32:091
2426899260,10cyclictest0-21swapper/111:55:051
2426899260,10cyclictest0-21swapper/110:10:011
161422260,2sleep2311ktimersoftd/208:00:272
2427399251,13cyclictest8809-21cpu09:00:112
2426899251,9cyclictest0-21swapper/110:22:341
2426899251,8cyclictest31036-21taskset11:03:171
2426899251,8cyclictest0-21swapper/108:18:021
2426899251,8cyclictest0-21swapper/108:18:021
2426899251,8cyclictest0-21swapper/107:36:371
2426899251,11cyclictest0-21swapper/110:17:271
2426899251,11cyclictest0-21swapper/107:55:561
2426899251,11cyclictest0-21swapper/107:46:171
2426899251,10cyclictest0-21swapper/111:14:251
2426899251,10cyclictest0-21swapper/109:37:491
2426899251,10cyclictest0-21swapper/108:50:551
2426899250,9cyclictest0-21swapper/110:41:021
2426899250,9cyclictest0-21swapper/110:41:021
2426899250,5cyclictest0-21swapper/110:25:541
2426899250,11cyclictest0-21swapper/112:20:091
2426899250,11cyclictest0-21swapper/111:39:471
2426899250,10cyclictest0-21swapper/109:45:051
2426899250,10cyclictest0-21swapper/109:21:491
2426899250,10cyclictest0-21swapper/108:32:061
2426899250,10cyclictest0-21swapper/108:21:461
2426099257,16cyclictest22991-21munin-run12:00:010
2426899241,10cyclictest0-21swapper/112:04:111
2426899240,9cyclictest0-21swapper/111:47:571
2426899240,9cyclictest0-21swapper/110:54:051
2426899240,9cyclictest0-21swapper/108:13:141
2426899240,8cyclictest0-21swapper/112:28:051
2426899231,10cyclictest0-21swapper/109:00:271
2426899230,8cyclictest0-21swapper/111:44:091
2426099231,11cyclictest23359-21idleruntime-cro08:20:010
2426099231,11cyclictest23359-21idleruntime-cro08:20:010
2427399224,2cyclictest87350irq/126-enp1s0-10:15:022
2427399221,12cyclictest0-21swapper/212:37:232
2427399221,11cyclictest32578-21perl12:20:152
2426899229,3cyclictest0-21swapper/110:55:151
2426899228,3cyclictest0-21swapper/111:06:071
2426899220,9cyclictest0-21swapper/112:15:451
2426899220,21cyclictest0-21swapper/109:27:171
2426899220,10cyclictest0-21swapper/111:30:431
2426899220,10cyclictest0-21swapper/109:42:371
24260992217,3cyclictest814-21grep07:25:260
2426099221,17cyclictest14554-21munin-run08:00:010
24260992211,3cyclictest13742-21grep11:35:240
2427399211,8cyclictest0-21swapper/209:50:352
2427399211,7cyclictest0-21swapper/211:55:332
24273992114,4cyclictest26428-21sed08:25:172
2426899210,8cyclictest0-21swapper/111:16:451
2426099215,4cyclictest18954-21munin-run08:10:010
2426099214,4cyclictest4772-21strings07:35:190
2426099213,11cyclictest12968-21diskmemload12:37:370
2426099213,10cyclictest0-21swapper/011:20:010
2426099212,11cyclictest30783-21/usr/sbin/munin09:50:150
24260992116,3cyclictest13577-21sh10:25:000
2426099211,5cyclictest0-21swapper/012:12:300
2426099211,11cyclictest20851-21ntp_states09:25:210
15332210,3sleep011-21rcuc/011:10:110
2427399206,6cyclictest30-21rcuc/209:55:162
2427399204,5cyclictest755-21rngd10:53:062
2427399202,5cyclictest30-21rcuc/211:50:222
2427399201,6cyclictest0-21swapper/210:55:432
24273992011,2cyclictest31537-21awk07:25:012
2426099206,8cyclictest0-21swapper/012:17:120
2426099204,14cyclictest25218-21awk12:05:010
2426099201,6cyclictest0-21swapper/011:43:040
2426099201,6cyclictest0-21swapper/009:30:430
2426099201,6cyclictest0-21swapper/009:30:430
2426099201,13cyclictest0-21swapper/010:54:060
2426099201,11cyclictest15396-21/usr/sbin/munin09:15:130
39612190,4sleep20-21swapper/210:00:222
2427399194,4cyclictest24616-21chrt12:00:262
2427399193,4cyclictest6557-21memory11:20:182
2427399192,14cyclictest15382-21grep10:25:272
2427399191,6cyclictest0-21swapper/210:45:172
24273991916,2cyclictest32-21ksoftirqd/208:35:012
24273991915,2cyclictest30-21rcuc/210:45:012
24273991915,2cyclictest30-21rcuc/210:45:012
2427399191,3cyclictest0-21swapper/210:37:342
2427399191,10cyclictest0-21swapper/209:15:182
2426099199,3cyclictest32024-21/usr/sbin/munin12:20:200
2426099199,2cyclictest3098-21grep07:30:270
2426099196,4cyclictest2596-21munin-run10:00:020
2426099191,7cyclictest0-21swapper/009:50:020
2426099191,5cyclictest0-21swapper/010:25:300
24260991912,4cyclictest10022-21grep10:15:160
2426099191,11cyclictest28926-21sshd10:59:330
2426099191,10cyclictest4399-21/usr/sbin/munin08:50:140
2426099191,10cyclictest4060-21/usr/sbin/munin12:30:180
2426099191,10cyclictest19175-21/usr/sbin/munin08:10:120
291822180,2sleep20-21swapper/207:18:222
2427399185,3cyclictest87350irq/126-enp1s0-11:10:022
2427399182,3cyclictest0-21swapper/209:39:562
2427399181,9cyclictest0-21swapper/211:30:142
2427399181,8cyclictest0-21swapper/210:32:182
2427399181,7cyclictest0-21swapper/210:24:192
2427399181,6cyclictest0-21swapper/211:17:552
2427399181,5cyclictest808-21snmpd08:57:222
2427399181,5cyclictest0-21swapper/210:18:282
24273991814,2cyclictest32-21ksoftirqd/207:35:232
2427399181,11cyclictest0-21swapper/212:16:312
2427399180,11cyclictest0-21swapper/212:28:372
2426099184,3cyclictest18546-21df_inode10:35:120
2426099183,4cyclictest0-21swapper/010:30:100
2426099181,8cyclictest5978-21/usr/sbin/munin11:20:120
2426099181,6cyclictest0-21swapper/010:48:010
24260991815,2cyclictest0-21swapper/009:35:140
2426099181,4cyclictest27386-21sshd08:25:340
24260991812,4cyclictest25558-21tr12:05:120
24260991812,3cyclictest2817-21/usr/sbin/munin10:00:230
2426099181,13cyclictest1860-21/usr/sbin/munin12:25:110
2426099181,10cyclictest0-21swapper/011:27:090
175902181,13sleep017772-21cstates09:20:120
107142181,2sleep00-21swapper/009:04:270
2427399171,8cyclictest0-21swapper/212:14:592
2427399171,6cyclictest13102-21cat09:10:012
2427399171,5cyclictest0-21swapper/207:53:342
2427399171,4cyclictest0-21swapper/211:44:102
24273991714,2cyclictest32-21ksoftirqd/208:00:002
24273991712,3cyclictest9089-21sendmail_mailtr11:25:232
24273991712,3cyclictest12758-21fschecks_time11:35:142
2427399171,15cyclictest0-21swapper/211:48:462
2427399171,15cyclictest0-21swapper/208:46:252
24273991711,3cyclictest4713-21gltestperf12:30:142
2427399171,10cyclictest0-21swapper/209:42:062
2426099177,6cyclictest0-21swapper/009:09:550
2426099174,4cyclictest9689-21grep07:45:270
2426099173,5cyclictest3181750kworker/u8:110:40:170
2426099173,5cyclictest3181750kworker/u8:110:40:170
2426099172,3cyclictest87350irq/126-enp1s0-11:00:240
2426099172,12cyclictest6757-21tr08:55:100
2426099171,7cyclictest27357-21/usr/sbin/munin07:15:230
2426099171,7cyclictest0-21swapper/010:11:580
24260991712,3cyclictest5029-21/usr/sbin/munin10:05:210
24260991712,3cyclictest29932-21crond08:35:010
24260991712,3cyclictest10033-21/usr/sbin/munin11:30:230
2426099171,13cyclictest7055-21ntp_states07:40:230
24260991711,3cyclictest10958-21perl07:50:150
24260991710,4cyclictest0-21swapper/011:45:270
231472170,2sleep20-21swapper/208:17:422
231472170,2sleep20-21swapper/208:17:422
213902171,2sleep20-21swapper/209:25:282
323542160,2sleep00-21swapper/008:40:030
2427399166,3cyclictest87350irq/126-enp1s0-07:40:262
2427399165,3cyclictest87350irq/126-enp1s0-08:05:142
2427399162,3cyclictest8183-21/usr/sbin/munin07:45:122
2427399161,8cyclictest0-21swapper/212:05:252
2427399161,6cyclictest0-21swapper/210:06:532
2427399161,6cyclictest0-21swapper/209:20:172
2427399161,6cyclictest0-21swapper/208:14:212
2427399161,4cyclictest763-21dbus-daemon11:00:212
24273991612,3cyclictest0-21swapper/208:50:072
2427399161,14cyclictest0-21swapper/209:11:332
2427399161,14cyclictest0-21swapper/208:20:582
24273991611,3cyclictest2978-21cat07:30:252
24273991611,3cyclictest29610-21nfs4_client09:45:192
24273991610,3cyclictest32275-21fschecks_time07:25:152
2427399160,6cyclictest0-21swapper/207:10:142
2426099165,3cyclictest29552-21/usr/sbin/munin07:20:160
2426099164,9cyclictest15906-21sed08:00:240
2426099161,9cyclictest0-21swapper/011:50:160
24260991611,3cyclictest24486-21cpu07:10:100
24260991610,3cyclictest14470-21swap09:10:250
24273991510,3cyclictest30468-21cat08:35:132
24273991510,3cyclictest1217-21sendmail_mailqu08:40:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional