You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 15:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Apr 19, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29662230162,55sleep10-21swapper/107:07:491
29432203166,24sleep30-21swapper/307:07:323
30862202165,24sleep20-21swapper/207:09:242
28052201164,25sleep00-21swapper/007:05:440
1628521820,5sleep3323399cyclictest10:05:013
412121580,3sleep2322999cyclictest08:20:262
2885921570,5sleep2322999cyclictest09:20:012
233221560,4sleep1322199cyclictest12:00:131
2425121010,2sleep20-21swapper/207:55:142
179882640,1sleep10-21swapper/112:35:161
179882640,1sleep10-21swapper/112:35:161
13192600,4sleep2322999cyclictest10:45:012
222942560,1sleep20-21swapper/211:30:202
102222510,2sleep2322999cyclictest12:15:272
66512480,1sleep16650-21grep12:10:121
283292460,1sleep20-21swapper/208:05:122
3213993711,13cyclictest18310-21munin-run10:10:010
3213993711,13cyclictest18310-21munin-run10:10:000
321399339,11cyclictest15484-21munin-run08:50:000
321399335,19cyclictest579-21systemd-journal11:08:540
322199321,11cyclictest0-21swapper/108:49:471
322199321,10cyclictest0-21swapper/108:05:031
322199320,24cyclictest0-21swapper/111:10:371
321399316,12cyclictest4169-21munin-run12:05:020
321399312,13cyclictest24473-21missed_timers11:35:200
322199301,10cyclictest16273-21cat12:30:191
321399300,15cyclictest26083-21/usr/sbin/munin08:00:230
322199291,19cyclictest12153-21proc_pri12:20:231
322199291,11cyclictest0-21swapper/111:42:151
322199290,13cyclictest0-21swapper/110:06:371
322199290,13cyclictest0-21swapper/110:06:371
322199290,12cyclictest0-21swapper/110:10:531
322199281,10cyclictest0-21swapper/107:41:061
322199280,26cyclictest0-21swapper/110:48:411
322199280,11cyclictest0-21swapper/109:26:051
322199280,10cyclictest0-21swapper/107:17:351
321399282,14cyclictest13301-21awk08:45:000
322199271,16cyclictest0-21swapper/107:12:291
322199271,10cyclictest31160-21smartctl08:10:151
322199270,17cyclictest0-21swapper/107:34:301
322199270,13cyclictest0-21swapper/109:40:011
321399273,12cyclictest29304-21munin-run10:35:010
321399272,12cyclictest10484-21munin-run07:25:010
321399271,13cyclictest16737-21munin-run11:20:010
322999261,6cyclictest0-21swapper/211:43:092
322199261,9cyclictest0-21swapper/108:33:571
3221992615,3cyclictest11315-21/usr/sbin/munin08:40:121
322199261,11cyclictest0-21swapper/108:16:031
322199261,11cyclictest0-21swapper/107:57:591
322199261,10cyclictest0-21swapper/108:02:271
321399263,11cyclictest24283-21diskmemload09:45:010
3213992613,3cyclictest17189-21kworker/0:112:35:170
3213992613,3cyclictest17189-21kworker/0:112:35:160
321399261,13cyclictest27102-21munin-run10:30:010
3213992610,7cyclictest8024-21latency_hist11:00:010
322999251,20cyclictest0-21swapper/211:16:262
322199251,12cyclictest0-21swapper/109:02:361
322199251,10cyclictest0-21swapper/111:45:031
322199251,10cyclictest0-21swapper/109:50:191
322199250,15cyclictest0-21swapper/110:44:371
322199250,10cyclictest0-21swapper/109:58:191
322199250,10cyclictest0-21swapper/108:21:151
321399256,8cyclictest15034-21taskset12:28:410
321399253,13cyclictest715-21uname11:55:180
321399252,5cyclictest87350irq/126-enp1s0-12:34:480
321399251,7cyclictest0-21swapper/010:43:290
321399251,12cyclictest17686-21munin-run08:55:000
323399248,9cyclictest0-21swapper/307:15:133
323399240,10cyclictest827-21runrttasks10:10:513
322999247,3cyclictest0-21swapper/209:42:192
322199241,5cyclictest0-21swapper/109:13:191
322199241,10cyclictest0-21swapper/112:09:251
322199241,10cyclictest0-21swapper/108:37:231
322199240,9cyclictest0-21swapper/111:15:271
322199240,8cyclictest0-21swapper/110:15:571
322199240,2cyclictest0-21swapper/110:33:491
322199240,10cyclictest0-21swapper/110:50:481
322199240,10cyclictest0-21swapper/108:53:271
321399245,16cyclictest20508-21munin-run10:15:000
321399244,7cyclictest17205-21latency_hist07:40:010
321399243,13cyclictest26326-21chrt09:13:100
321399242,9cyclictest23033-21/usr/sbin/munin07:50:250
321399241,19cyclictest24283-21diskmemload10:50:160
321399241,15cyclictest7978-21fschecks_count09:45:160
321399241,12cyclictest11721-21awk09:55:010
321399240,13cyclictest8367-21ls08:30:270
323399239,3cyclictest30952-21irqstats11:50:183
323399231,15cyclictest0-21swapper/311:40:123
3229992320,2cyclictest0-21swapper/210:49:192
322999231,4cyclictest0-21swapper/208:10:142
322199237,2cyclictest0-21swapper/109:45:231
322199231,9cyclictest0-21swapper/111:02:551
322199231,9cyclictest0-21swapper/107:48:191
322199231,9cyclictest0-21swapper/107:48:181
322199231,9cyclictest0-21swapper/107:35:181
322199231,3cyclictest0-21swapper/112:25:031
322199231,19cyclictest6929-21users09:40:251
322199231,12cyclictest10484-21munin-run07:25:011
322199231,11cyclictest0-21swapper/110:28:471
322199231,10cyclictest24283-21diskmemload11:36:471
322199231,10cyclictest0-21swapper/107:53:531
322199230,9cyclictest0-21swapper/112:19:351
322199230,9cyclictest0-21swapper/108:29:031
322199230,17cyclictest0-21swapper/109:32:311
322199230,10cyclictest0-21swapper/111:58:071
322199230,10cyclictest0-21swapper/111:05:431
321399238,6cyclictest14033-21grep11:10:260
321399231,17cyclictest0-21swapper/012:24:460
321399231,16cyclictest0-21swapper/009:20:220
321399230,8cyclictest24283-21diskmemload11:30:170
3233992215,2cyclictest0-21swapper/310:40:163
322999225,3cyclictest0-21swapper/211:11:582
322999221,9cyclictest0-21swapper/208:52:072
322999221,8cyclictest23667-21munin-run07:55:012
3229992212,7cyclictest5717-21systemd-journal10:54:572
322999221,13cyclictest0-21swapper/209:57:542
322999221,12cyclictest0-21swapper/210:27:312
322199227,3cyclictest0-21swapper/111:31:191
322199222,3cyclictest0-21swapper/108:58:111
322199222,3cyclictest0-21swapper/108:58:111
322199221,18cyclictest16112-21munin-run10:05:011
322199221,10cyclictest0-21swapper/109:05:341
322199220,8cyclictest0-21swapper/111:26:171
322199220,16cyclictest0-21swapper/107:26:191
321399228,5cyclictest0-21swapper/011:43:080
321399227,7cyclictest0-21swapper/009:16:460
321399224,6cyclictest11169-21cat08:40:010
321399222,8cyclictest25291-21kworker/u8:212:10:120
321399222,6cyclictest2183650sshd09:01:550
321399221,9cyclictest193150kworker/u8:012:09:350
321399221,7cyclictest9856-21ntp_states12:15:230
321399221,6cyclictest0-21swapper/009:30:450
321399220,10cyclictest30584-21/usr/sbin/munin10:35:220
321399220,10cyclictest23272-21sh10:20:130
190922220,4sleep123-21ksoftirqd/111:25:021
190922220,4sleep123-21ksoftirqd/111:25:021
323399218,2cyclictest15789-21cpuspeed_turbos08:50:123
323399217,2cyclictest5537-21pmu-power12:05:213
323399212,7cyclictest29884-21forks10:35:163
323399211,7cyclictest31782-21hddtemp_smartct09:25:173
3233992114,2cyclictest0-21swapper/311:00:263
3233992113,2cyclictest0-21swapper/307:45:133
3233992113,2cyclictest0-21swapper/307:45:133
323399211,2cyclictest87550irq/128-enp1s0-09:51:363
323399211,17cyclictest18568-21grep12:35:213
323399211,17cyclictest18568-21grep12:35:213
323399211,14cyclictest0-21swapper/311:08:473
3233992111,3cyclictest26468-21fschecks_count08:00:143
323399211,10cyclictest9850-21ntp_states12:15:223
3233992110,4cyclictest0-21swapper/310:55:103
322999215,3cyclictest87650irq/129-enp1s0-08:15:152
322999213,3cyclictest87650irq/129-enp1s0-10:15:142
322999212,4cyclictest30785-21kworker/u8:209:35:152
322999211,8cyclictest6590-21/usr/sbin/munin12:10:202
322999211,7cyclictest0-21swapper/210:36:222
322999211,6cyclictest755-21rngd12:35:202
322999211,6cyclictest755-21rngd12:35:202
322999211,6cyclictest0-21swapper/209:11:012
322999211,4cyclictest0-21swapper/211:48:282
322999211,17cyclictest0-21swapper/210:30:172
322999211,10cyclictest16055-21hddtemp_smartct12:30:172
322199217,3cyclictest0-21swapper/110:58:341
322199216,3cyclictest0-21swapper/111:53:471
322199211,17cyclictest30380-21ls10:35:191
322199210,9cyclictest0-21swapper/109:24:151
322199210,8cyclictest0-21swapper/109:18:371
321399216,4cyclictest0-21swapper/011:26:340
321399215,4cyclictest14127-21/usr/sbin/munin10:00:120
321399213,3cyclictest7636-21switchtime07:15:240
321399213,3cyclictest13060-21cpuspeed_turbos07:30:120
321399213,10cyclictest3139-21/usr/sbin/munin09:35:210
321399212,11cyclictest20733-21/usr/sbin/munin10:15:220
321399211,8cyclictest0-21swapper/009:59:230
321399211,14cyclictest0-21swapper/011:47:230
321399211,12cyclictest2525-21/usr/sbin/munin08:20:150
242532210,4sleep19-21rcu_preempt10:20:241
323399206,3cyclictest4499-21crond08:25:003
323399202,5cyclictest25321-21irqstats09:10:173
323399201,6cyclictest12611-21hddtemp_smartct09:55:173
3233992014,3cyclictest9769-21gltestperf08:35:163
3233992012,3cyclictest0-21swapper/308:29:363
3233992011,3cyclictest0-21swapper/308:14:573
322999202,2cyclictest87650irq/129-enp1s0-11:02:362
322999201,4cyclictest8033-21sshd07:16:552
322999201,4cyclictest12873-21cat07:30:012
322999201,11cyclictest0-21swapper/208:38:032
321399208,4cyclictest0-21swapper/010:46:060
321399205,6cyclictest0-21swapper/011:50:040
321399205,12cyclictest31578-21fschecks_time09:25:160
321399203,5cyclictest30256-21sh08:10:010
321399203,3cyclictest20275-21if_err_enp1s007:45:180
321399203,3cyclictest20275-21if_err_enp1s007:45:180
321399202,15cyclictest24283-21diskmemload11:24:590
321399202,15cyclictest24283-21diskmemload11:24:590
321399201,6cyclictest0-21swapper/008:15:230
3213992012,5cyclictest5329-40dnf07:14:230
3213992012,5cyclictest25853-21latency_hist08:00:020
321399201,11cyclictest17296-21/usr/sbin/munin07:40:170
323399199,3cyclictest4233-21irqstats07:10:173
323399199,3cyclictest11698-21needreboot07:25:183
323399198,2cyclictest0-21swapper/312:25:263
323399191,9cyclictest28686-21/usr/sbin/munin09:20:013
323399191,6cyclictest900-21tail08:15:153
323399191,6cyclictest8351-21irqstats09:45:183
323399191,6cyclictest28625-21smartctl11:45:173
323399191,6cyclictest28580-21/usr/sbin/munin10:30:243
323399191,6cyclictest21369-21smart_sda09:00:243
323399191,6cyclictest17299-21df_inode11:20:153
323399191,6cyclictest17299-21df_inode11:20:153
323399191,3cyclictest0-21swapper/311:11:593
323399191,3cyclictest0-21swapper/310:45:163
323399191,10cyclictest28294-21/usr/sbin/munin08:05:173
3233991910,3cyclictest0-21swapper/311:15:273
322999193,4cyclictest11633-21expr08:40:112
322999193,3cyclictest0-21swapper/211:37:512
322999191,9cyclictest0-21swapper/210:08:332
322999191,9cyclictest0-21swapper/210:08:332
322999191,9cyclictest0-21swapper/207:20:132
322999191,8cyclictest31357-21uname11:50:232
322999191,6cyclictest9735-21/usr/sbin/munin09:50:202
322999191,6cyclictest7542-21/usr/sbin/munin09:45:222
322999191,6cyclictest7053-21open_inodes10:55:202
322999191,6cyclictest22315-21/usr/sbin/munin09:05:172
322999191,6cyclictest16945-21sshd11:20:012
322999191,6cyclictest16945-21sshd11:20:012
322999191,5cyclictest19156-21/usr/sbin/munin11:25:252
322999191,5cyclictest12901-21/usr/sbin/munin07:30:142
322999191,5cyclictest0-21swapper/209:34:292
322999191,15cyclictest15258-21grep10:00:202
322999191,10cyclictest0-21swapper/211:58:202
321399195,4cyclictest8827-21gltestperf11:00:150
321399194,7cyclictest30486-21/usr/sbin/munin08:10:230
323399189,2cyclictest0-21swapper/309:35:123
323399181,6cyclictest0-21swapper/310:50:213
323399181,3cyclictest0-21swapper/312:10:173
323399181,14cyclictest401ktimersoftd/308:40:233
323399180,5cyclictest0-21swapper/309:30:243
323399180,16cyclictest10488-21perf07:25:013
322999187,3cyclictest0-21swapper/210:20:132
322999184,3cyclictest87550irq/128-enp1s0-08:00:142
322999182,7cyclictest13181-21/usr/sbin/munin12:25:232
322999182,2cyclictest87550irq/128-enp1s0-08:45:292
322999181,6cyclictest4860-21fschecks_time12:05:152
322999181,6cyclictest0-21swapper/212:24:452
322999181,5cyclictest2472-21expr12:00:132
322999181,5cyclictest19408-21latency10:10:182
322999181,4cyclictest0-21swapper/209:28:332
3229991813,3cyclictest0-21swapper/207:44:172
3229991810,6cyclictest2265-21kworker/u8:109:03:262
321399184,11cyclictest18728-21uname08:55:180
321399184,11cyclictest18728-21uname08:55:170
321399182,8cyclictest0-21swapper/008:25:160
3213991811,4cyclictest11054-21diskstats07:25:130
240752180,2sleep30-21swapper/309:07:323
323399174,6cyclictest39-21rcuc/308:34:213
323399173,4cyclictest41-21ksoftirqd/309:41:443
323399173,4cyclictest0-21swapper/311:56:313
323399172,3cyclictest2241-21cpuspeed_turbos12:00:123
323399171,6cyclictest41-21ksoftirqd/307:42:103
323399171,5cyclictest0-21swapper/311:26:593
3233991713,2cyclictest19496-21users08:55:263
3233991713,2cyclictest19496-21users08:55:263
323399171,2cyclictest0-21swapper/310:25:133
323399171,14cyclictest0-21swapper/311:35:083
323399171,14cyclictest0-21swapper/309:24:553
323399171,14cyclictest0-21swapper/307:55:043
3233991711,3cyclictest22929-21unixbench_singl11:30:273
323399171,10cyclictest0-21swapper/310:19:563
323399170,11cyclictest0-21swapper/312:33:013
322999171,9cyclictest0-21swapper/208:30:292
322999171,6cyclictest20095-21gltestperf07:45:162
322999171,6cyclictest20095-21gltestperf07:45:162
322999171,6cyclictest0-21swapper/208:27:362
322999171,6cyclictest0-21swapper/207:10:272
322999171,5cyclictest0-21swapper/211:09:252
322999171,15cyclictest0-21swapper/208:56:402
322999171,15cyclictest0-21swapper/208:56:402
322999171,14cyclictest0-21swapper/207:35:562
321399171,8cyclictest808-21snmpd09:07:520
248452171,2sleep30-21swapper/310:24:293
323399161,14cyclictest0-21swapper/312:20:493
323399161,14cyclictest0-21swapper/308:49:473
323399161,14cyclictest0-21swapper/307:32:053
3233991611,3cyclictest21812-21cpuspeed_turbos07:50:133
3233991611,3cyclictest15099-21/usr/sbin/munin07:35:233
323399160,14cyclictest0-21swapper/310:05:133
323399160,14cyclictest0-21swapper/310:05:133
3229991610,3cyclictest29237-21expr09:20:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional