You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 05:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Jul 18, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114682241163,25sleep20-21swapper/219:09:242
115132204167,25sleep30-21swapper/319:09:583
113392204167,24sleep10-21swapper/119:07:431
114062201164,25sleep00-21swapper/019:08:350
11613996258,3cyclictest0-21swapper/200:08:292
105122570,3sleep00-21swapper/000:00:010
105122570,3sleep00-21swapper/000:00:010
11613995451,2cyclictest0-21swapper/220:10:542
320572530,4sleep339-21rcuc/323:35:123
194032530,7sleep341-21ksoftirqd/321:50:263
1160399378,14cyclictest4708-21munin-run21:20:000
1161399362,13cyclictest4618-21smart_sda22:30:232
1161399331,23cyclictest0-21swapper/221:25:262
1160399338,12cyclictest21400-21idleruntime-cro00:25:010
1161399321,22cyclictest0-21swapper/221:19:462
1161399321,22cyclictest0-21swapper/219:42:012
1161399321,12cyclictest30652-21sed22:15:252
1161399311,25cyclictest0-21swapper/223:42:082
1161399311,24cyclictest0-21swapper/220:49:382
1161399311,14cyclictest0-21swapper/222:40:592
1161399311,13cyclictest0-21swapper/223:22:182
1161399311,10cyclictest0-21swapper/221:21:262
1161399310,23cyclictest0-21swapper/219:51:172
1161399301,17cyclictest0-21swapper/220:30:262
1161399301,12cyclictest0-21swapper/221:40:192
1161399301,11cyclictest0-21swapper/221:35:162
1161399301,11cyclictest0-21swapper/221:35:152
1161399301,10cyclictest11709-21cat20:20:182
1161399301,10cyclictest0-21swapper/200:27:502
1161399300,23cyclictest0-21swapper/219:14:542
1161399291,17cyclictest0-21swapper/219:27:252
1161399291,12cyclictest0-21swapper/221:49:592
1161399291,12cyclictest0-21swapper/221:11:362
1161399290,18cyclictest0-21swapper/219:55:292
1160799291,17cyclictest27012-21needreboot00:35:191
1160799291,10cyclictest0-21swapper/121:14:471
1161399281,12cyclictest0-21swapper/220:40:282
1161399281,12cyclictest0-21swapper/200:22:162
1161399281,11cyclictest0-21swapper/222:21:492
1161399281,10cyclictest0-21swapper/220:35:542
189782270,3sleep011-21rcuc/000:17:150
1161399271,12cyclictest0-21swapper/222:35:112
1161399271,12cyclictest0-21swapper/220:27:342
1161399271,12cyclictest0-21swapper/219:35:232
1161399271,11cyclictest0-21swapper/221:55:192
1161399271,10cyclictest0-21swapper/220:06:052
1161399270,9cyclictest0-21swapper/219:30:092
1161399270,12cyclictest0-21swapper/200:34:232
1161399270,11cyclictest0-21swapper/200:01:502
1161399270,10cyclictest0-21swapper/223:07:092
1160799271,16cyclictest0-21swapper/119:50:131
1160399279,15cyclictest13788-21perf19:15:000
1161399261,9cyclictest0-21swapper/222:25:152
1161399261,9cyclictest0-21swapper/200:16:422
1161399261,15cyclictest23258-21if_enp1s022:00:172
1161399261,12cyclictest0-21swapper/222:08:192
1161399261,11cyclictest0-21swapper/223:59:382
1161399261,11cyclictest0-21swapper/223:59:382
1161399261,11cyclictest0-21swapper/222:45:572
1161399261,11cyclictest0-21swapper/222:45:572
1161399261,11cyclictest0-21swapper/219:49:172
1161399261,11cyclictest0-21swapper/219:16:542
1161399261,10cyclictest32669-21diskmemload23:45:242
1161399261,10cyclictest0-21swapper/223:52:582
1161399261,10cyclictest0-21swapper/200:10:382
1161399260,12cyclictest0-21swapper/222:54:132
1161399260,11cyclictest0-21swapper/223:37:302
1160799260,18cyclictest0-21swapper/123:29:181
1160399268,15cyclictest2524-21awk21:15:000
39422250,4sleep33944-21gltestperf22:30:143
1161399251,10cyclictest0-21swapper/221:07:382
1161399250,17cyclictest0-21swapper/220:59:022
1161399250,11cyclictest0-21swapper/220:01:332
1160799251,8cyclictest0-21swapper/121:59:271
1162099245,10cyclictest0-21swapper/322:10:123
1161399241,12cyclictest0-21swapper/221:51:452
1161399241,10cyclictest18898-21cat19:25:012
1161399240,13cyclictest0-21swapper/200:39:012
1161399240,10cyclictest0-21swapper/221:00:302
1160799241,18cyclictest0-21swapper/120:56:351
1160399247,10cyclictest0-21swapper/023:15:310
1162099232,11cyclictest25062-21df22:05:123
1160799231,3cyclictest16065-21tr21:45:121
1160799231,19cyclictest8168-21ntp_kernel_pll_21:25:211
1160799231,16cyclictest0-21swapper/119:18:051
1160799231,16cyclictest0-21swapper/119:12:121
1160799230,21cyclictest0-21swapper/123:24:371
313932220,2sleep10-21swapper/121:05:161
1161399222,3cyclictest0-21swapper/220:15:252
1161399221,9cyclictest0-21swapper/223:12:302
1161399220,12cyclictest0-21swapper/222:11:572
1160799229,11cyclictest7348-21kworker/u8:120:44:021
11607992215,4cyclictest22532-21head20:45:171
1160799221,3cyclictest14198-21open_files20:25:231
1160799221,3cyclictest0-21swapper/123:15:261
1160799221,18cyclictest6888-21hddtemp_smartct23:50:171
1160799221,16cyclictest0-21swapper/123:31:231
1160799220,16cyclictest0-21swapper/100:29:121
1160399227,3cyclictest15098-21basename20:30:000
1160399221,13cyclictest0-21swapper/022:28:520
1162099215,2cyclictest12057-21pmu-power20:20:223
1162099214,14cyclictest39-21rcuc/319:35:183
1162099211,10cyclictest23081-21cat23:15:013
1162099211,10cyclictest0-21swapper/319:10:223
1161399217,3cyclictest0-21swapper/220:53:382
1161399210,13cyclictest0-21swapper/223:33:462
1161399210,10cyclictest0-21swapper/222:55:452
1160799217,11cyclictest23331-21uname22:00:181
1160799214,10cyclictest755-21rngd20:10:251
1160799211,9cyclictest9912-21perl21:30:151
11607992117,3cyclictest0-21swapper/121:35:471
11607992117,3cyclictest0-21swapper/121:35:471
11607992115,3cyclictest25571-21grep20:50:271
1160799211,19cyclictest0-21swapper/121:01:561
1160799211,14cyclictest0-21swapper/100:20:461
1160799211,11cyclictest4662-21df_inode20:05:131
1160799211,11cyclictest26593-21memory19:40:181
1160799211,11cyclictest15312-21acpi20:30:111
1160799211,10cyclictest20108-21netstat19:25:181
11607992110,3cyclictest13056-21ls00:05:121
1160799210,15cyclictest87650irq/129-enp1s0-00:33:301
1160399219,3cyclictest2654-21kernelversion23:40:170
1160399216,7cyclictest87350irq/126-enp1s0-19:40:020
1160399211,12cyclictest0-21swapper/022:30:140
1160399210,10cyclictest81ktimersoftd/021:50:260
140042201,2sleep3808-21snmpd22:53:073
1162099208,4cyclictest15040-21/usr/sbin/munin00:10:253
1162099208,4cyclictest0-21swapper/319:55:193
1162099207,4cyclictest868-21fschecks_count21:10:153
1162099207,2cyclictest6174-21gltestperf22:35:143
1162099207,2cyclictest0-21swapper/321:30:183
1162099204,4cyclictest23604-21turbostat.cron00:30:023
1162099204,3cyclictest87650irq/129-enp1s0-21:55:263
1162099203,4cyclictest9229-21kernelversion23:55:183
1162099203,4cyclictest9229-21kernelversion23:55:183
1162099202,8cyclictest19702-21/usr/sbin/munin20:40:133
1162099202,6cyclictest23530-21df_inode23:15:123
1162099202,10cyclictest20427-21needreboot00:20:203
1162099201,9cyclictest0-21swapper/323:25:423
1162099201,7cyclictest20347-21smart_sda19:25:223
1162099201,7cyclictest0-21swapper/322:20:443
1162099201,11cyclictest0-21swapper/321:43:443
1162099201,10cyclictest6294-21cat23:50:133
1162099201,10cyclictest28298-21latency_hist21:00:013
1162099201,10cyclictest23867-21latency_hist20:50:013
1162099201,10cyclictest0-21swapper/319:48:363
1161399203,3cyclictest0-21swapper/221:33:342
1161399201,5cyclictest25143-21cat23:19:572
1160799209,5cyclictest28775-21kworker/u8:122:30:281
1160799204,7cyclictest0-21swapper/122:40:451
1160799202,9cyclictest18125-21/usr/sbin/munin21:50:171
1160799201,8cyclictest0-21swapper/122:19:521
1160799201,4cyclictest13722-21unixbench_multi22:50:261
11607992013,2cyclictest0-21swapper/119:20:091
11607992012,2cyclictest0-21swapper/100:01:501
1160799201,17cyclictest0-21swapper/123:49:031
1160799201,14cyclictest0-21swapper/119:35:491
1160799201,10cyclictest18767-21cat23:05:021
1160799201,10cyclictest0-21swapper/121:20:141
1160799200,18cyclictest0-21swapper/120:16:211
1160799200,15cyclictest0-21swapper/119:46:411
1160799200,15cyclictest0-21swapper/100:11:161
1160799200,14cyclictest0-21swapper/119:31:521
1160399209,3cyclictest30701-21/usr/sbin/munin21:05:170
1160399205,7cyclictest28496-21/usr/sbin/munin21:00:230
1160399205,4cyclictest0-21swapper/022:19:360
1160399203,3cyclictest0-21swapper/000:26:290
1160399202,9cyclictest1194250kworker/u8:223:02:390
1160399202,15cyclictest5351-21cut21:20:140
1160399201,8cyclictest22518-21/usr/sbin/munin22:00:190
11603992013,4cyclictest9468-21tr21:30:120
1160399201,2cyclictest0-21swapper/021:45:150
1160399201,11cyclictest2110-21/usr/sbin/munin20:00:220
1160399201,11cyclictest0-21swapper/023:10:280
1160399201,11cyclictest0-21swapper/021:27:020
1160399200,9cyclictest0-21swapper/022:50:570
257832191,2sleep00-21swapper/000:34:530
1162099197,3cyclictest23589-21ntp_states22:00:233
1162099193,9cyclictest41-21ksoftirqd/322:25:243
1162099192,10cyclictest0-21swapper/323:06:553
1162099191,7cyclictest28011-21idleruntime-cro00:40:023
1162099191,5cyclictest26063-21crond20:55:013
1162099191,4cyclictest24622-21if_err_enp2s000:30:173
11620991913,3cyclictest17854-21diskstats20:35:143
1162099191,15cyclictest29840-21cpu23:30:113
1162099191,10cyclictest28496-21/usr/sbin/munin21:00:173
1162099191,10cyclictest27602-21sshd19:44:303
1162099191,10cyclictest0-21swapper/323:45:503
1162099191,10cyclictest0-21swapper/322:40:213
1162099190,9cyclictest0-21swapper/321:06:343
1160799196,4cyclictest3811-21sshd20:01:321
11607991916,2cyclictest0-21swapper/122:12:321
11607991915,3cyclictest0-21swapper/120:20:361
1160799191,4cyclictest1475-21df22:25:141
11607991914,3cyclictest2912-21open_files23:40:221
1160799191,2cyclictest0-21swapper/123:36:201
1160799191,17cyclictest0-21swapper/122:56:361
1160799191,16cyclictest0-21swapper/122:35:351
1160799191,16cyclictest0-21swapper/122:05:481
11607991911,2cyclictest0-21swapper/100:15:021
1160799191,10cyclictest20182-21sed23:05:251
1160799191,10cyclictest1480-21users19:55:281
1160799190,6cyclictest0-21swapper/123:13:431
1160399196,5cyclictest87350irq/126-enp1s0-21:55:270
1160399196,4cyclictest0-21swapper/023:08:430
1160399194,11cyclictest31064-21needreboot19:50:190
1160399193,8cyclictest87550irq/128-enp1s0-20:10:110
1160399193,7cyclictest5574-21/usr/sbin/munin22:35:250
1160399193,5cyclictest11-21rcuc/022:20:130
1160399193,10cyclictest1326550kworker/u8:021:43:400
1160399192,11cyclictest0-21swapper/000:37:440
1160399191,8cyclictest81ktimersoftd/023:55:000
1160399191,8cyclictest2099-1kworker/0:2H21:35:210
1160399191,8cyclictest2099-1kworker/0:2H21:35:210
1160399191,4cyclictest0-21swapper/023:21:230
1160399191,4cyclictest0-21swapper/022:05:230
1160399191,3cyclictest28407-21sed22:10:260
1160399191,10cyclictest4318-21/usr/sbin/munin20:05:250
1160399191,10cyclictest0-21swapper/023:31:430
1160399191,10cyclictest0-21swapper/023:27:460
96432180,2sleep00-21swapper/022:43:380
1162099185,4cyclictest12461-21sshd00:03:203
1162099184,11cyclictest39-21rcuc/320:10:153
1162099183,3cyclictest18857-21sh19:25:023
1162099183,12cyclictest9702-21strings20:15:193
1162099183,12cyclictest39-21rcuc/323:40:193
1162099181,9cyclictest16170-21grep19:15:283
1162099181,9cyclictest0-21swapper/323:22:423
1162099181,5cyclictest17911-21awk21:50:023
11620991812,3cyclictest21288-21/usr/sbin/munin19:30:243
1162099181,11cyclictest0-21swapper/320:25:143
1162099181,10cyclictest0-21swapper/300:17:313
1161399186,2cyclictest0-21swapper/223:27:182
1161399181,11cyclictest0-21swapper/223:00:192
1160799181,8cyclictest0-21swapper/122:23:171
1160799181,16cyclictest0-21swapper/122:46:071
1160799181,16cyclictest0-21swapper/122:46:071
1160799181,16cyclictest0-21swapper/121:40:301
1160799180,4cyclictest0-21swapper/123:56:031
1160799180,4cyclictest0-21swapper/123:56:031
1160799180,17cyclictest0-21swapper/121:16:371
1160799180,10cyclictest18574-21ntpq20:35:221
1160399187,3cyclictest17074-21sshd20:32:310
1160399185,3cyclictest0-21swapper/000:05:280
1160399182,9cyclictest0-21swapper/000:01:290
1160399182,4cyclictest10542-21perl22:45:150
1160399182,4cyclictest10542-21perl22:45:150
1160399182,3cyclictest5828-21NetworkManager23:48:060
1160399181,6cyclictest0-21swapper/023:35:020
1160399181,4cyclictest19087-21/usr/sbin/munin19:25:150
1160399181,15cyclictest0-21swapper/020:59:480
1160399180,9cyclictest0-21swapper/022:56:220
317402171,3sleep30-21swapper/319:51:153
193492170,2sleep00-21swapper/020:38:320
172142170,3sleep30-21swapper/320:34:213
1162099179,5cyclictest0-21swapper/321:35:123
1162099179,5cyclictest0-21swapper/321:35:123
1162099172,4cyclictest2855-21cut21:15:133
1162099172,12cyclictest7130-21/usr/sbin/munin21:25:173
1162099171,9cyclictest0-21swapper/322:47:543
1162099171,9cyclictest0-21swapper/322:47:543
1162099171,7cyclictest5276-21df_inode21:20:143
1162099171,6cyclictest30035-21memory22:15:183
1162099171,6cyclictest13745-21memory00:05:193
1162099171,3cyclictest15236-21kernelversion22:55:193
1162099171,10cyclictest0-21swapper/323:01:153
1160399176,10cyclictest0-21swapper/019:35:020
1160399172,9cyclictest0-21swapper/000:11:470
1160399171,8cyclictest808-21snmpd19:45:240
1160399171,8cyclictest0-21swapper/020:45:260
1160399171,4cyclictest16887-21/usr/sbin/munin19:20:190
11603991712,3cyclictest32044-21latency_hist19:55:030
11603991712,3cyclictest19963-21ls20:40:120
11603991711,3cyclictest24196-21cpuspeed_turbos20:50:130
106212171,2sleep00-21swapper/020:19:250
1162099161,6cyclictest4318-21/usr/sbin/munin20:05:243
11620991611,3cyclictest2988-21wc20:00:173
1160399162,6cyclictest87350irq/126-enp1s0-20:20:190
11603991610,3cyclictest26206-21cut19:40:150
11603991510,3cyclictest14326-21head19:15:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional