You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 05:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 27, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2088210088,8sleep00-21swapper/019:09:470
208028976,9sleep20-21swapper/219:09:412
179828659,9sleep30-21swapper/319:06:013
183828464,16sleep10-21swapper/119:06:311
219799280,0cyclictest0-21swapper/300:17:503
219499270,26cyclictest0-21swapper/200:00:082
219499270,24cyclictest0-21swapper/200:22:442
219099264,12cyclictest0-21swapper/100:35:131
219499250,0cyclictest0-21swapper/221:53:402
219099251,23cyclictest20093-21hddtemp_smartct19:45:171
219499240,24cyclictest0-21swapper/223:52:582
219499230,5cyclictest0-21swapper/221:32:392
219499230,23cyclictest0-21swapper/200:33:142
2190992321,1cyclictest25-21ksoftirqd/122:27:001
2190992316,1cyclictest25-21ksoftirqd/100:17:021
2190992220,1cyclictest25-21ksoftirqd/123:51:111
219099220,1cyclictest27570-21diskmemload23:21:481
219099220,19cyclictest0-21swapper/123:28:541
219799210,20cyclictest0-21swapper/319:50:133
219499211,1cyclictest0-21swapper/220:50:132
219499210,20cyclictest0-21swapper/221:41:002
2190992111,1cyclictest25-21ksoftirqd/123:03:271
219499200,20cyclictest0-21swapper/222:44:592
219499200,1cyclictest0-21swapper/220:25:132
2190992018,1cyclictest25-21ksoftirqd/121:54:071
2190992013,4cyclictest25-21ksoftirqd/122:35:001
219499190,19cyclictest0-21swapper/222:38:292
2190991917,2cyclictest0-21swapper/122:45:521
219099190,19cyclictest0-21swapper/100:11:291
219499180,17cyclictest0-21swapper/221:11:492
219099186,1cyclictest141rcu_preempt22:10:341
2188991812,5cyclictest25401-21ssh22:25:170
219799176,10cyclictest0-21swapper/300:09:523
2194991717,0cyclictest0-21swapper/200:18:032
2194991710,6cyclictest5889-21wc23:30:232
219499170,1cyclictest0-21swapper/223:27:222
219499170,17cyclictest0-21swapper/223:15:122
219499170,14cyclictest0-21swapper/200:07:102
219099170,5cyclictest141rcu_preempt21:30:281
218899175,6cyclictest17085-21munin-run22:40:000
218899173,12cyclictest23020-21swap23:40:260
219799162,1cyclictest141rcu_preempt21:44:503
2194991616,0cyclictest0-21swapper/223:21:442
2194991616,0cyclictest0-21swapper/223:08:532
219499160,2cyclictest0-21swapper/223:49:122
219499160,16cyclictest0-21swapper/221:35:342
219499160,16cyclictest0-21swapper/200:12:142
2190991614,2cyclictest25-21ksoftirqd/122:22:321
2190991612,2cyclictest25-21ksoftirqd/121:50:011
218899169,5cyclictest30372-21grep23:25:280
218899162,2cyclictest141rcu_preempt22:49:370
218899160,16cyclictest0-21swapper/022:02:150
2197991512,2cyclictest39-21ksoftirqd/323:50:123
2194991510,3cyclictest2559-21grep19:10:132
219499150,1cyclictest0-21swapper/223:03:132
219499150,12cyclictest0-21swapper/222:15:342
219499150,12cyclictest0-21swapper/219:40:132
219099153,3cyclictest141rcu_preempt22:37:471
2190991513,2cyclictest241rcuc/121:21:491
2190991513,1cyclictest25-21ksoftirqd/100:26:331
2190991512,2cyclictest25-21ksoftirqd/122:50:161
219099151,1cyclictest141rcu_preempt20:05:211
2190991511,1cyclictest25-21ksoftirqd/120:50:141
219099150,1cyclictest0-21swapper/122:15:231
2188991514,1cyclictest0-21swapper/022:53:440
219799140,13cyclictest17727-21ls22:20:223
219499141,1cyclictest141rcu_preempt23:10:322
2194991410,3cyclictest23360-21head19:50:252
2194991410,3cyclictest20817-21grep20:55:142
2194991410,2cyclictest15456-21ssh22:00:152
219499140,14cyclictest0-21swapper/222:10:522
219499140,14cyclictest0-21swapper/221:17:282
219499140,13cyclictest18054-21turbostat20:50:002
219099149,1cyclictest25-21ksoftirqd/100:31:131
219099146,4cyclictest25-21ksoftirqd/123:07:321
219099145,5cyclictest198-21jbd2/sda2-800:00:001
219099144,3cyclictest141rcu_preempt22:56:201
219099141,7cyclictest27570-21diskmemload21:13:401
2190991412,2cyclictest25-21ksoftirqd/100:02:381
2190991411,1cyclictest141rcu_preempt00:05:241
2190991410,3cyclictest10075-21tr19:25:151
219099140,2cyclictest6977-21runrttasks23:30:551
218899149,3cyclictest13668-21cat19:30:250
218899143,9cyclictest18275-21sendmail22:59:590
218899143,8cyclictest15556-21memory19:35:210
218899143,6cyclictest19052-21munin-run19:45:010
218899142,6cyclictest31450-21munin-run22:09:590
218899140,1cyclictest0-21swapper/023:19:050
218899140,14cyclictest0-21swapper/000:36:110
219799139,3cyclictest932-21tr22:30:163
219799139,3cyclictest15177-21ssh21:21:003
219799133,10cyclictest0-21swapper/322:58:533
2197991310,2cyclictest3253-21chrt21:33:423
2197991310,2cyclictest29700-21sort20:05:183
2197991310,2cyclictest26144-21ls22:25:223
2197991310,2cyclictest25772-21chrt00:01:243
2197991310,2cyclictest17018-21grep00:35:133
2197991310,2cyclictest16664-21head20:45:163
2197991310,2cyclictest12918-21head19:30:173
219799130,12cyclictest26856-21grep22:45:173
219499139,3cyclictest23733-21cut22:05:132
2194991310,2cyclictest346-21rm00:25:172
2194991310,2cyclictest25881-21grep19:55:252
219499130,3cyclictest0-21swapper/223:56:052
219499130,1cyclictest0-21swapper/222:47:512
219499130,1cyclictest0-21swapper/222:23:492
219499130,12cyclictest0-21swapper/220:15:162
219099139,3cyclictest17600-21grep20:45:251
219099138,4cyclictest25-21ksoftirqd/123:40:131
219099138,2cyclictest25-21ksoftirqd/121:55:161
219099137,2cyclictest25-21ksoftirqd/123:10:141
219099132,10cyclictest25-21ksoftirqd/121:15:401
2190991312,1cyclictest25-21ksoftirqd/122:41:421
219099131,1cyclictest141rcu_preempt22:04:411
219099131,12cyclictest25-21ksoftirqd/123:35:161
2190991310,2cyclictest25716-21cut21:05:151
2190991310,2cyclictest21912-21tr19:50:151
2190991310,2cyclictest13331-21dump-pmu-power21:40:011
219099130,13cyclictest0-21swapper/121:41:311
219099130,12cyclictest0-21swapper/120:40:001
218899135,6cyclictest13878-21ssh21:40:140
218899131,6cyclictest12243-21munin-run23:35:010
2188991313,0cyclictest0-21swapper/022:23:070
2188991312,1cyclictest0-21swapper/021:10:310
2188991310,2cyclictest8118-21tail19:20:160
2188991310,2cyclictest21923-21grep19:50:150
2188991310,2cyclictest18982-21grep20:50:170
2188991310,2cyclictest13063-21grep21:20:180
218899130,6cyclictest6175-21idleruntime-cro21:55:000
218899130,11cyclictest0-21swapper/022:42:420
218899130,0cyclictest0-21swapper/021:35:560
219799129,2cyclictest790-21tr22:10:213
219799129,2cyclictest7183-21grep20:25:163
219799129,2cyclictest5417-21ssh21:15:223
219799129,2cyclictest4294-21tr20:20:143
219799129,2cyclictest294052sleep323:05:353
219799129,2cyclictest29299-21grep23:45:013
219799129,2cyclictest26201-21grep21:05:183
219799129,2cyclictest2271-21tr22:50:123
219799129,2cyclictest19075-21grep20:50:173
219799129,2cyclictest18502-21tail22:40:183
219799129,2cyclictest14405-21grep23:55:153
219799129,2cyclictest136092sleep322:18:173
219799129,2cyclictest12288-21head23:15:173
219799129,2cyclictest10503-21head19:25:173
219799128,3cyclictest5801-21grep21:35:173
219799120,12cyclictest0-21swapper/323:49:173
219799120,12cyclictest0-21swapper/323:10:153
219499129,2cyclictest22904-21cut21:45:172
219499129,2cyclictest21679-21grep23:40:162
219499129,2cyclictest15084-21grep19:35:182
219499129,2cyclictest10674-21tr22:55:152
219499128,3cyclictest4145-21grep20:20:142
219499120,11cyclictest10887-21grep19:25:202
219099129,2cyclictest8511-21cron20:30:001
219099129,2cyclictest5666-21sort19:15:181
219099129,2cyclictest5666-21sort19:15:181
219099129,2cyclictest3165-21grep19:10:161
219099129,2cyclictest25370-21grep19:55:191
219099129,2cyclictest23787-21head21:00:151
219099129,2cyclictest14234-21cut20:40:161
219099127,4cyclictest25-21ksoftirqd/120:55:141
219099126,2cyclictest141rcu_preempt23:48:311
219099120,3cyclictest141rcu_preempt22:05:131
219099120,11cyclictest9046-21grep19:20:251
219099120,11cyclictest25975-21grep00:20:271
218899129,2cyclictest9173-21cut22:15:200
218899129,2cyclictest903-21ssh22:30:150
218899129,2cyclictest7307-21ssh21:55:150
218899129,2cyclictest32242-21grep22:10:150
218899129,2cyclictest28100-21grep23:05:180
218899129,2cyclictest27388-21head20:00:180
218899129,2cyclictest22001-21fgrep20:55:220
218899129,2cyclictest21978-21cron23:59:590
218899129,2cyclictest20916-21seq00:18:010
218899129,2cyclictest16035-21tr20:45:130
218899128,3cyclictest4997-21grep23:10:250
218899122,9cyclictest5769-21python21:15:270
218899122,4cyclictest9210-21grep00:30:170
218899121,8cyclictest22499-21/usr/sbin/munin00:00:220
218899120,11cyclictest30049-21cut20:05:200
218899120,11cyclictest21647-21ls23:20:270
219799118,2cyclictest8745-21ssh21:55:293
219799118,2cyclictest8140-21head00:10:203
219799118,2cyclictest7899-21df19:20:143
219799118,2cyclictest6339-21grep19:15:243
219799118,2cyclictest6339-21grep19:15:233
219799118,2cyclictest592-21ssh23:27:293
219799118,2cyclictest3395-21ssh00:26:323
219799118,2cyclictest30030-21ssh00:23:353
219799118,2cyclictest28628-21taskset20:02:303
219799118,2cyclictest28077-21cat23:25:013
219799118,2cyclictest23107-21grep21:00:123
219799118,2cyclictest21156-21grep21:25:153
219799118,2cyclictest18688-21ssh23:00:093
219799118,2cyclictest16227-21grep19:35:283
219799118,2cyclictest14895-21ssh22:00:053
219799118,2cyclictest1461-21taskset21:51:223
219799118,2cyclictest10116-21cat00:30:263
219799117,3cyclictest24829-21head22:05:203
219799110,10cyclictest8716-21sh23:32:173
219799110,10cyclictest30499-21ssh21:10:433
219799110,10cyclictest20138-21cat19:45:183
219499119,1cyclictest3885-21sed22:50:242
219499118,2cyclictest8713-21sh21:55:282
219499118,2cyclictest29646-21df20:05:172
219499118,2cyclictest27418-21tail20:00:182
219499118,2cyclictest25788-21cat22:25:202
219499118,2cyclictest22700-21grep21:25:282
219499118,2cyclictest19540-21tr19:45:152
219499118,2cyclictest18683-21head00:35:262
219499118,2cyclictest13990-21ssh21:20:252
219499118,2cyclictest12800-21grep20:35:282
219499118,2cyclictest10773-21taskset20:33:322
219499117,2cyclictest14104-21wc23:35:222
219499113,6cyclictest1175-21cstates22:30:162
219099118,2cyclictest4779-21grep20:20:171
219099118,2cyclictest27150-21df20:00:161
219099118,2cyclictest26736-21ssh21:28:291
219099118,2cyclictest11480-21ssh23:15:111
218899118,2cyclictest9559-21tail20:30:160
218899118,2cyclictest7245-21ssh00:10:150
218899118,2cyclictest4264-21tr20:20:140
218899118,2cyclictest32566-21cat20:10:220
218899118,2cyclictest31787-21ssh21:30:390
218899118,2cyclictest28529-21ssh00:22:240
218899118,2cyclictest25924-21ssh21:46:450
218899118,2cyclictest23747-21grep21:00:150
218899118,2cyclictest23015-21ssh21:25:340
218899118,2cyclictest20482-21cat23:00:260
218899118,2cyclictest1780-21ssh00:06:220
218899118,2cyclictest14411-21grep23:35:250
218899118,2cyclictest13662-21grep20:40:140
218899111,7cyclictest4537-21turbostat.cron19:15:000
2188991110,1cyclictest0-21swapper/000:28:100
218899110,10cyclictest32098-21ssh23:45:410
219799108,1cyclictest31886-21expr20:10:173
219799108,1cyclictest1841-21cat20:15:153
219799108,1cyclictest17334-21ls19:40:153
219799108,1cyclictest15333-21sshd20:40:533
219799108,1cyclictest12675-21appstreamcli23:35:013
219799108,1cyclictest12054-21cat20:35:203
219799107,2cyclictest9814-21cat20:30:193
219799107,2cyclictest22193-21basename21:45:133
219799107,2cyclictest21168-21df20:55:153
219799107,2cyclictest10143-21ssh22:35:203
219799106,3cyclictest4808-21cat19:15:013
219799100,9cyclictest24959-21tune2fs19:55:163
219499108,1cyclictest87982sleep219:20:222
219499108,1cyclictest13528-21chrt20:40:112
219499108,1cyclictest12367-21grep19:30:142
219499107,2cyclictest24311-21head21:00:202
219499103,5cyclictest27553-21cron21:10:002
219099108,1cyclictest18483-21sed19:40:251
219099107,2cyclictest9022-21grep20:30:141
219099107,2cyclictest31695-21tr20:10:171
219099107,2cyclictest12893-21head19:30:161
219099100,9cyclictest3114-21sed20:15:251
218899108,1cyclictest9747-21chrt19:25:050
218899108,1cyclictest6703-21cat23:50:170
218899108,1cyclictest260992sleep019:56:350
218899108,1cyclictest254622sleep021:05:130
218899108,1cyclictest11432-21expr20:35:160
218899107,2cyclictest6661-21tr20:25:140
218899107,2cyclictest5285-21expr19:15:160
218899107,2cyclictest5285-21expr19:15:160
218899107,2cyclictest3634-21chrt20:18:330
21949997,1cyclictest1053-21chrt20:13:562
21909997,1cyclictest15858-21fgrep19:35:241
21889996,2cyclictest20519-21ls19:45:210
21949987,1cyclictest5128-21awk19:15:162
21949987,1cyclictest5128-21awk19:15:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional