You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 13:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 18, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1184129443,47sleep10-21swapper/119:06:461
1204629279,9sleep00-21swapper/019:09:260
1183328846,38sleep30-21swapper/319:06:403
1195628644,38sleep20-21swapper/219:08:162
191762670,0sleep00-21swapper/022:54:370
12181993616,19cyclictest0-21swapper/223:15:112
12181993014,15cyclictest0-21swapper/220:45:132
12181992723,4cyclictest0-21swapper/222:22:332
1218299240,3cyclictest0-21swapper/322:59:183
1218299236,2cyclictest0-21swapper/323:41:543
12182992221,1cyclictest0-21swapper/321:56:503
12181992218,4cyclictest0-21swapper/222:33:532
1217999220,18cyclictest0-21swapper/023:53:390
12182992016,3cyclictest5906-21aten2_r7power_c23:25:153
12181992015,4cyclictest26906-21sensors00:15:232
1218199200,19cyclictest0-21swapper/223:10:132
12181991915,4cyclictest0-21swapper/221:49:332
12181991914,4cyclictest0-21swapper/222:03:032
1218199190,2cyclictest0-21swapper/222:48:092
12180991919,0cyclictest0-21swapper/121:14:391
12180991919,0cyclictest0-21swapper/100:30:081
1217999192,13cyclictest1142-21runrttasks21:34:380
72762180,0sleep00-21swapper/000:04:290
1218299186,12cyclictest0-21swapper/300:32:213
1218199185,13cyclictest0-21swapper/219:17:092
12181991816,2cyclictest0-21swapper/221:53:132
12181991814,4cyclictest0-21swapper/200:11:232
12181991814,1cyclictest0-21swapper/223:20:332
12181991813,4cyclictest12064-21ssh00:06:432
1218199181,17cyclictest55650irq/127-enp2s0-22:10:242
1218199180,1cyclictest0-21swapper/200:25:242
12180991816,1cyclictest0-21swapper/123:20:161
1218099181,15cyclictest5225-21diskmemload23:48:241
1218099180,18cyclictest0-21swapper/121:25:531
1217999183,14cyclictest0-21swapper/022:34:220
1218199174,13cyclictest0-21swapper/219:43:192
1218199170,17cyclictest0-21swapper/222:51:102
1218199170,17cyclictest0-21swapper/222:35:012
1218199170,14cyclictest0-21swapper/222:17:302
1218199170,14cyclictest0-21swapper/222:17:302
1218099175,1cyclictest0-21swapper/100:37:061
1218099170,1cyclictest0-21swapper/122:36:181
1218099170,11cyclictest0-21swapper/120:50:151
1218099170,0cyclictest0-21swapper/123:38:441
1217999171,13cyclictest14498-21/usr/sbin/munin21:15:150
12182991615,1cyclictest0-21swapper/300:01:003
1218299160,16cyclictest0-21swapper/321:27:383
12181991613,3cyclictest0-21swapper/200:23:032
12181991612,4cyclictest0-21swapper/221:36:542
1218199160,4cyclictest141rcu_preempt23:06:242
1218099160,3cyclictest0-21swapper/121:36:431
1218199155,10cyclictest0-21swapper/220:37:092
12181991513,2cyclictest0-21swapper/222:08:532
12181991512,3cyclictest0-21swapper/223:35:232
12181991512,1cyclictest0-21swapper/200:37:162
12181991511,4cyclictest0-21swapper/221:12:532
1218199150,15cyclictest0-21swapper/223:29:172
1218099153,8cyclictest5225-21diskmemload23:10:171
1218099150,0cyclictest0-21swapper/121:45:101
12179991515,0cyclictest0-21swapper/021:25:150
1217999151,10cyclictest24976-21apt-get22:00:150
1217999150,12cyclictest0-21swapper/019:25:150
12182991411,2cyclictest2716-21wc00:20:243
12182991410,3cyclictest6803-21grep20:05:133
12182991410,3cyclictest16896-21sendmail19:20:003
12182991410,2cyclictest24260-21tr21:40:153
1218299140,3cyclictest0-21swapper/322:17:533
1218299140,3cyclictest0-21swapper/322:17:533
1218299140,2cyclictest141rcu_preempt22:20:273
1218299140,13cyclictest0-21swapper/323:11:163
1218199147,7cyclictest0-21swapper/220:52:192
12181991411,3cyclictest0-21swapper/223:31:132
12181991411,3cyclictest0-21swapper/222:28:532
12181991410,3cyclictest6704-21ssh23:45:002
1218099144,3cyclictest225-21systemd-journal22:15:131
1218099144,3cyclictest225-21systemd-journal22:15:131
1218099141,7cyclictest0-21swapper/121:42:381
12180991412,1cyclictest8011-21ssh21:30:171
12180991411,2cyclictest8895-21wc22:10:001
12180991411,2cyclictest20958-21cat20:35:011
12180991411,2cyclictest20746-21sed19:25:201
12180991410,3cyclictest16321-21ssh21:15:281
1218099140,1cyclictest0-21swapper/100:10:141
1217999149,3cyclictest12671-21cut19:10:160
1217999145,4cyclictest1510-21apache223:03:170
1217999141,11cyclictest26144-21ssh23:55:270
12179991410,3cyclictest10006-21cron22:30:000
286582130,0chrt0-21swapper/122:20:481
1218299139,3cyclictest17334-21cut00:10:153
1218299138,3cyclictest4432-21cut20:00:153
1218299138,3cyclictest29991-21wget23:20:153
1218299138,3cyclictest20348-21ssh22:35:203
1218299134,5cyclictest225-21systemd-journal22:35:003
12182991310,2cyclictest3441-21wc19:55:283
12182991310,2cyclictest30500-21grep23:00:273
12182991310,2cyclictest26845-21sed22:00:273
12182991310,2cyclictest16398-21tail23:30:273
12182991310,2cyclictest10769-21grep20:10:283
1218299130,2cyclictest0-21swapper/322:10:213
1218299130,13cyclictest0-21swapper/322:08:143
1218299130,12cyclictest4255-21grep22:45:173
1218299130,12cyclictest23151-21grep21:20:163
1218299130,11cyclictest5225-21diskmemload23:05:253
1218199139,3cyclictest23767-21wc23:55:012
1218199139,3cyclictest18117-21basename00:30:142
1218199139,2cyclictest20002-21tr19:25:162
12181991311,2cyclictest0-21swapper/221:42:232
12181991310,2cyclictest304-21cut00:00:142
12181991310,2cyclictest21038-21grep22:55:172
12181991310,2cyclictest18324-21cron20:30:002
12181991310,2cyclictest17414-21ssh21:16:202
1218099139,3cyclictest17211-21cat20:25:211
1218099139,3cyclictest15234-21ssh00:09:071
1218099139,3cyclictest10197-21tr20:10:221
12180991312,1cyclictest0-21swapper/123:05:121
12180991311,1cyclictest0-21swapper/100:28:561
12180991310,2cyclictest9114-21grep21:50:161
12180991310,2cyclictest30424-21cat23:00:271
12180991310,2cyclictest25844-21sed22:00:181
12180991310,2cyclictest22270-21hostname23:15:151
12180991310,2cyclictest15145-21grep23:30:181
12180991310,2cyclictest10761-21tr22:10:191
12180991310,1cyclictest5225-21diskmemload00:23:531
1218099130,13cyclictest0-21swapper/122:52:501
1218099130,12cyclictest10610-21basename22:30:151
1217999139,3cyclictest28637-21cat19:40:260
1217999139,3cyclictest26209-21tr20:45:150
1217999139,3cyclictest13189-21sh23:10:150
1217999137,4cyclictest1528-21apache222:10:150
1217999132,5cyclictest0-21swapper/000:10:000
1217999131,8cyclictest5996-21aten2_r7power_p21:10:110
12179991310,2cyclictest9698-21grep20:10:190
12179991310,2cyclictest31746-21grep23:20:270
12179991310,2cyclictest2747-21cat22:05:260
12179991310,2cyclictest26726-21ssh00:15:210
12179991310,2cyclictest24726-21grep21:20:270
12179991310,2cyclictest22898-21grep23:35:140
12179991310,2cyclictest22159-21grep22:55:260
12179991310,2cyclictest17846-21grep20:25:280
12179991310,2cyclictest17456-21cron00:30:000
1218299129,2cyclictest39-21ksoftirqd/321:45:213
1218299129,2cyclictest28843-21sed22:40:233
1218299129,2cyclictest21668-21grep23:15:123
1218299129,2cyclictest20243-21tail20:30:273
1218299129,2cyclictest15671-21ssh21:35:153
1218299129,2cyclictest14600-21cut20:20:183
1218299129,2cyclictest14600-21cut20:20:183
1218299128,3cyclictest31976-21grep19:50:123
1218299128,3cyclictest13142-21tail19:10:183
1218299127,3cyclictest8838-21wc21:30:253
12182991212,0cyclictest0-21swapper/300:09:483
1218199129,3cyclictest0-21swapper/222:40:232
1218199129,3cyclictest0-21swapper/221:30:232
1218199129,2cyclictest8959-21wc23:45:252
1218199129,2cyclictest28744-21tr23:00:152
1218199129,2cyclictest14450-21sort20:20:182
1218199129,2cyclictest14450-21sort20:20:172
1218199129,2cyclictest12227-21head20:15:182
1218199122,9cyclictest56550irq/124-enp0s3121:05:152
12181991212,0cyclictest0-21swapper/221:29:202
1218199120,6cyclictest0-21swapper/219:20:132
1218199120,12cyclictest0-21swapper/221:57:062
1218099129,3cyclictest0-21swapper/120:35:011
1218099129,2cyclictest7614-21cat23:25:271
1218099129,2cyclictest4013-21cat22:25:271
1218099129,2cyclictest32154-21grep19:50:131
1218099129,2cyclictest31216-21grep19:45:251
1218099129,2cyclictest14140-21tr20:20:161
1218099129,2cyclictest14140-21tr20:20:161
1218099128,3cyclictest6754-21ssh22:46:281
1218099128,3cyclictest28340-21grep22:40:181
12180991212,0cyclictest0-21swapper/123:44:011
1218099120,1cyclictest25615-21kworker/1:022:58:141
1218099120,11cyclictest26127-21grep00:15:161
1217999129,2cyclictest23645-21wc21:40:000
1217999129,2cyclictest20031-21cat20:30:270
1217999129,2cyclictest18551-21sort22:15:160
1217999129,2cyclictest18551-21sort22:15:160
1217999129,2cyclictest16909-21awk19:20:000
1217999129,2cyclictest15316-21head23:30:170
1217999128,2cyclictest4029-21cron20:00:000
1217999124,4cyclictest18262-21needreboot00:10:200
1217999120,1cyclictest0-21swapper/020:40:140
1218299119,1cyclictest0-21swapper/321:15:013
1218299118,2cyclictest7226-21ssh21:10:193
1218299118,2cyclictest6650-21ssh22:27:213
1218299118,2cyclictest31377-21cut20:55:183
1218299118,2cyclictest28447-21hostname20:50:173
1218299118,2cyclictest28129-21wc20:50:003
1218299118,2cyclictest25056-21ssh00:15:013
1218299118,2cyclictest24428-21tr23:55:153
1218299118,2cyclictest23928-21ls19:30:253
1218299118,2cyclictest22834-21taskset23:35:123
1218299118,2cyclictest22439-21wc20:35:213
1218299118,2cyclictest21873-21sh19:30:003
1218299118,2cyclictest19542-21cut19:25:003
1218299118,2cyclictest17538-21wc23:50:263
1218299118,2cyclictest15163-21ssh22:51:293
1218299118,2cyclictest14470-21ssh00:27:423
1218299118,2cyclictest10292-21ssh23:46:173
1218299117,3cyclictest23861-21tr20:40:163
1218299116,3cyclictest2756-21runrttasks21:04:013
1218299114,5cyclictest15351-21apache200:37:323
1218299110,1cyclictest0-21swapper/321:50:093
1218199119,1cyclictest0-21swapper/223:55:172
1218199118,2cyclictest7677-21cat20:05:182
1218199118,2cyclictest23830-21tr20:40:162
1218199118,2cyclictest20498-21chrt20:32:302
1218199118,2cyclictest1773-21ls19:55:002
1218199118,2cyclictest13731-21find19:10:232
1218199118,2cyclictest10828-21chrt20:10:382
1218199113,4cyclictest225-21systemd-journal19:47:382
1218199111,9cyclictest56550irq/124-enp0s3119:55:272
1218099119,1cyclictest17054-21sed19:20:001
1218099118,2cyclictest2835-21cron21:04:591
1218099118,2cyclictest25538-21head23:55:221
1218099118,2cyclictest25501-21tail19:35:171
1218099118,2cyclictest24249-21chrt19:32:221
1218099118,2cyclictest23289-21ssh23:54:461
1218099118,2cyclictest2242-21cat19:55:161
1218099118,2cyclictest14128-21sed19:10:271
1218099115,2cyclictest225-21systemd-journal20:45:001
1218099112,8cyclictest17713-21/usr/sbin/munin21:55:221
12180991110,1cyclictest0-21swapper/121:20:081
1218099110,10cyclictest0-21swapper/120:15:141
1217999118,2cyclictest9649-21grep21:50:210
1217999118,2cyclictest953-21ssh00:20:010
1217999118,2cyclictest6989-21tr20:05:140
1217999118,2cyclictest5254-21ssh21:48:030
1217999118,2cyclictest5216-21cat20:00:190
1217999118,2cyclictest4655-21grep22:45:200
1217999118,2cyclictest3231-21ssh23:42:170
1217999118,2cyclictest31532-21taskset19:49:120
1217999118,2cyclictest28952-21ssh00:35:570
1217999118,2cyclictest27886-21expr22:40:150
1217999118,2cyclictest27498-21ntp_kernel_pll_22:20:220
1217999118,2cyclictest25668-21ssh23:17:100
1217999118,2cyclictest21221-21ssh21:57:300
1217999118,2cyclictest19652-21grep22:35:150
1217999118,2cyclictest17643-21grep19:20:140
1217999118,2cyclictest14127-21grep20:20:160
1217999118,2cyclictest14127-21grep20:20:160
1217999118,2cyclictest13601-21sendmail-msp20:20:000
1217999117,3cyclictest335-21cron21:00:000
12179991110,1cyclictest0-21swapper/019:30:240
1217999110,6cyclictest0-21swapper/023:05:120
1217999110,4cyclictest5216-21idleruntime-cro21:10:000
1217999110,10cyclictest28959-21ssh21:42:470
1218299108,1cyclictest4904-21chrt21:06:083
1218299108,1cyclictest29000-21seq19:42:073
1218299108,1cyclictest26492-21taskset19:35:383
1218299107,2cyclictest30226-21grep19:45:163
1218299107,2cyclictest11516-21cut20:15:143
1218199108,1cyclictest30901-21grep20:55:162
1218199108,1cyclictest25181-21expr19:35:142
1218199108,1cyclictest22721-21ssh21:20:082
1218199107,2cyclictest22307-21appstreamcli19:30:012
1218199100,9cyclictest56550irq/124-enp0s3121:00:232
1218199100,9cyclictest0-21swapper/220:00:152
1218099107,2cyclictest6990-21tr20:05:141
1218099107,2cyclictest580-21df00:00:141
1218099107,2cyclictest4633-21cat20:00:161
1218099107,2cyclictest31111-21expr20:55:161
1218099107,2cyclictest27967-21cat19:40:181
1218099105,4cyclictest19240-21taskset19:21:441
1217999108,1cyclictest6405-21ssh23:25:160
1217999108,1cyclictest32235-21expr19:50:130
1217999108,1cyclictest21223-21aten2_r7power_e20:35:130
1217999108,1cyclictest1686-21cut21:00:210
1217999107,2cyclictest17900-21rm00:30:100
1217999103,3cyclictest225-21systemd-journal23:45:010
1217999101,4cyclictest0-21swapper/020:50:140
121829997,1cyclictest17983-21chrt20:25:413
121809997,1cyclictest5262-21cat21:10:011
121809996,2cyclictest25912-21taskset20:45:041
121799985,2cyclictest24735-21apt-get19:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional