You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 22:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Apr 18, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
616129078,8sleep20-21swapper/207:06:082
615429068,17sleep00-21swapper/007:06:010
640228659,21sleep10-21swapper/107:09:161
635228471,9sleep30-21swapper/307:08:383
655799280,28cyclictest0-21swapper/211:37:152
655399275,21cyclictest0-21swapper/112:35:131
655799260,26cyclictest0-21swapper/212:01:152
655799240,1cyclictest0-21swapper/212:05:202
655799240,0cyclictest0-21swapper/209:10:002
655799230,23cyclictest0-21swapper/211:59:352
655399230,1cyclictest0-21swapper/111:44:031
655799220,22cyclictest0-21swapper/212:24:412
655799220,19cyclictest0-21swapper/211:18:062
655799210,21cyclictest0-21swapper/209:56:202
655799210,0cyclictest0-21swapper/210:43:272
655799200,20cyclictest0-21swapper/212:38:122
655799200,20cyclictest0-21swapper/211:32:492
655799200,20cyclictest0-21swapper/209:37:052
655399200,1cyclictest0-21swapper/111:53:571
655999190,19cyclictest0-21swapper/312:36:323
655799190,19cyclictest0-21swapper/209:46:312
655399190,15cyclictest0-21swapper/109:10:171
655199191,13cyclictest0-21swapper/010:45:120
655999180,1cyclictest0-21swapper/312:07:053
655999180,14cyclictest0-21swapper/310:58:513
655799180,1cyclictest0-21swapper/212:27:562
6553991817,1cyclictest0-21swapper/109:36:361
655399180,17cyclictest0-21swapper/112:19:491
655199180,1cyclictest0-21swapper/011:09:190
655199180,18cyclictest0-21swapper/011:35:120
89402170,0sleep30-21swapper/309:15:183
655999170,3cyclictest0-21swapper/311:30:103
655999170,16cyclictest0-21swapper/311:04:293
655799170,3cyclictest0-21swapper/210:03:152
6553991715,2cyclictest0-21swapper/111:39:311
655399170,16cyclictest0-21swapper/109:43:431
655199174,7cyclictest7528-21awk09:15:010
655999165,3cyclictest141rcu_preempt11:25:273
655799160,8cyclictest0-21swapper/210:15:002
6553991615,1cyclictest0-21swapper/111:15:281
6553991615,1cyclictest0-21swapper/110:08:101
655399160,1cyclictest0-21swapper/112:14:281
655399160,1cyclictest0-21swapper/110:23:491
655199160,16cyclictest0-21swapper/009:23:310
655999153,5cyclictest0-21swapper/310:20:573
6557991514,1cyclictest0-21swapper/211:51:402
6557991514,1cyclictest0-21swapper/210:33:342
6557991510,2cyclictest24668-21head11:20:212
655799150,15cyclictest0-21swapper/209:19:392
655399150,1cyclictest0-21swapper/111:10:131
655399150,15cyclictest0-21swapper/110:29:061
655199151,5cyclictest9140-21ssh09:35:140
6551991511,3cyclictest0-21swapper/009:25:080
6551991511,3cyclictest0-21swapper/009:25:070
6551991510,3cyclictest22919-21ssh11:20:010
655199150,0cyclictest0-21swapper/011:42:350
6559991411,1cyclictest0-21swapper/312:22:223
6559991410,3cyclictest26424-21cron12:20:013
655999140,10cyclictest0-21swapper/310:25:133
655799141,6cyclictest4737-21cron10:50:002
6557991412,1cyclictest0-21swapper/210:27:382
6557991411,2cyclictest9910-21cron08:24:592
6557991411,2cyclictest440-21cat08:05:012
6557991410,2cyclictest26607-21grep07:50:172
655799140,1cyclictest0-21swapper/210:09:502
655799140,1cyclictest0-21swapper/209:40:102
655799140,1cyclictest0-21swapper/207:28:362
655799140,13cyclictest0-21swapper/211:04:252
6553991414,0cyclictest0-21swapper/109:59:151
655399141,3cyclictest0-21swapper/111:09:501
6553991413,1cyclictest0-21swapper/107:10:191
6553991411,2cyclictest10700-21wc07:15:251
6553991410,3cyclictest8477-21head08:20:151
6553991410,3cyclictest29882-21grep10:45:141
655399140,4cyclictest225-21systemd-journal12:28:541
655399140,13cyclictest0-21swapper/107:55:131
655199149,3cyclictest18161-21cut08:40:150
655199148,3cyclictest17246-21cron08:39:590
6551991410,2cyclictest2669-21sed12:05:190
655199140,12cyclictest5097-21awk08:14:590
139392140,0sleep00-21swapper/007:25:010
655999139,3cyclictest25798-21wc07:50:013
655999139,2cyclictest18386-21sed09:20:283
6559991313,0cyclictest0-21swapper/310:18:573
6559991310,2cyclictest9453-21grep07:15:163
6559991310,2cyclictest1962-21grep09:30:193
6559991310,2cyclictest1894-21cut09:50:163
6559991310,2cyclictest10357-21grep12:10:143
6559991310,1cyclictest0-21swapper/309:40:123
655999130,12cyclictest13379-21grep10:35:183
655999130,0cyclictest0-21swapper/310:32:113
655799139,3cyclictest7216-21grep10:50:262
655799139,3cyclictest7216-21grep10:50:252
655799138,3cyclictest12305-21idleruntime-cro08:30:002
6557991311,1cyclictest0-21swapper/211:11:372
6557991310,3cyclictest487-21avahi-daemon09:29:132
6557991310,3cyclictest487-21avahi-daemon09:29:122
6557991310,2cyclictest14424-21grep08:30:272
655799130,1cyclictest0-21swapper/209:32:472
655799130,11cyclictest0-21swapper/212:16:082
655399139,3cyclictest7503-21cat11:30:001
655399139,3cyclictest21531-21cron11:00:001
655399137,1cyclictest0-21swapper/112:06:551
655399133,0cyclictest141rcu_preempt09:26:401
655399133,0cyclictest141rcu_preempt09:26:401
655399131,2cyclictest0-21swapper/110:30:361
6553991310,2cyclictest6838-21wc08:15:271
6553991310,2cyclictest15917-21cut08:35:171
6553991310,2cyclictest12957-21wc12:30:201
6553991310,2cyclictest12957-21wc12:30:201
655399130,1cyclictest0-21swapper/112:20:281
655199139,3cyclictest8438-21ssh11:30:130
655199132,6cyclictest19638-21munin-run12:35:010
655199132,6cyclictest19638-21munin-run12:35:000
6551991313,0cyclictest0-21swapper/010:59:230
6551991310,2cyclictest31337-21tune2fs08:00:170
6551991310,2cyclictest28930-21wc10:45:010
6551991310,2cyclictest2620-21grep09:50:190
6551991310,2cyclictest14261-21tr07:25:160
6551991310,2cyclictest11090-21awk12:30:000
655199130,13cyclictest0-21swapper/009:46:020
655999129,2cyclictest7811-21ssh11:10:193
655999129,2cyclictest5493-21rm11:09:313
655999129,2cyclictest5298-21cut10:50:143
655999129,2cyclictest5298-21cut10:50:133
655999129,2cyclictest31190-21df08:00:163
655999129,2cyclictest30416-21cat09:05:203
655999129,2cyclictest23369-21tr11:20:123
655999129,2cyclictest10898-21wc09:35:263
655999128,3cyclictest22544-21grep08:50:153
655999128,3cyclictest11904-21grep07:20:163
655999125,4cyclictest0-21swapper/308:30:163
6559991211,1cyclictest0-21swapper/310:12:553
655799129,2cyclictest9174-21cut07:15:142
655799129,2cyclictest7328-21grep07:10:152
655799129,2cyclictest31556-21sed11:05:172
655799129,2cyclictest31527-21cron09:10:002
655799129,2cyclictest24308-21sleep09:25:012
655799129,2cyclictest2230-21ssh11:45:222
655799129,2cyclictest12822-21ntpq10:15:212
655799127,4cyclictest0-21swapper/212:13:012
655799121,4cyclictest18745-21sshd07:35:142
655799120,1cyclictest0-21swapper/212:33:072
655799120,1cyclictest0-21swapper/212:33:062
655799120,12cyclictest0-21swapper/210:58:442
655399129,2cyclictest5308-21grep10:50:141
655399129,2cyclictest5308-21grep10:50:131
655399129,2cyclictest3832-21tail10:10:171
655399129,2cyclictest3042-21grep08:10:131
655399129,2cyclictest30134-21tr09:05:181
655399129,2cyclictest2672-21grep09:30:261
655399129,2cyclictest23237-21sh07:45:001
655399129,2cyclictest20163-21tr08:45:151
655399129,2cyclictest19029-21tr07:35:151
655399129,2cyclictest17571-21grep08:40:121
655399129,2cyclictest14456-21grep08:30:281
655399129,2cyclictest12815-21tr10:35:161
655399129,2cyclictest12454-21chrt09:17:161
6553991211,1cyclictest30998-21ssh12:02:531
6553991211,1cyclictest25-21ksoftirqd/109:54:061
655399120,12cyclictest0-21swapper/110:17:161
655399120,10cyclictest0-21swapper/110:41:331
655399120,10cyclictest0-21swapper/109:24:441
655199129,2cyclictest9397-21cut11:50:140
655199129,2cyclictest5863-21ssh10:30:230
655199129,2cyclictest31137-21turbostat11:24:590
655199129,2cyclictest27778-21ssh12:20:170
655199129,2cyclictest25300-21cron12:00:000
655199129,2cyclictest24708-21basename08:55:150
655199129,2cyclictest24026-21grep11:00:260
655199129,2cyclictest19543-21grep10:00:170
655199129,2cyclictest19443-21grep07:35:170
655199129,2cyclictest19364-21cron10:20:000
655199129,2cyclictest18316-21head09:40:160
655199129,2cyclictest10933-21sed08:25:170
655199129,2cyclictest10030-21tail09:15:270
655199128,3cyclictest5231-21ssh10:50:120
655199128,3cyclictest5231-21ssh10:50:120
655199128,3cyclictest20899-21cut12:35:170
655199128,3cyclictest2007-21tr09:30:200
655199128,3cyclictest11565-21grep09:55:220
655199122,9cyclictest14524-21perf11:15:000
655199122,8cyclictest19953-21aten2_r7power_e10:20:120
655199120,0cyclictest0-21swapper/012:00:020
655999119,1cyclictest28501-21grep07:55:143
655999119,1cyclictest194352sleep308:42:253
655999118,2cyclictest9510-21cat11:50:153
655999118,2cyclictest8416-21grep08:20:143
655999118,2cyclictest7390-21ssh12:27:103
655999118,2cyclictest32402-21tr09:10:163
655999118,2cyclictest28104-21uname10:05:173
655999118,2cyclictest27083-21ssh09:25:403
655999118,2cyclictest27083-21ssh09:25:393
655999118,2cyclictest25966-21wget12:00:133
655999118,2cyclictest25036-21tr11:40:153
655999118,2cyclictest21564-21grep08:45:263
655999118,2cyclictest2120-21ssh10:47:583
655999118,2cyclictest19391-21ssh11:17:223
655999118,2cyclictest19323-21ssh11:36:143
655999118,2cyclictest19106-21cat10:00:143
655999118,2cyclictest17365-21ssh11:55:083
655999118,2cyclictest14690-21ssh12:31:093
655999118,2cyclictest14690-21ssh12:31:083
655999118,2cyclictest14382-21expr07:25:153
655999118,2cyclictest11126-21cut08:25:193
655999118,2cyclictest10549-21cat09:55:153
655999117,3cyclictest1573-21head11:45:163
655999117,2cyclictest18470-21idleruntime-cro07:35:013
655999114,4cyclictest28040-21unixbench_singl09:45:363
655999111,4cyclictest0-21swapper/310:41:543
655799119,2cyclictest291irq_work/207:55:132
655799118,2cyclictest6447-21grep08:15:212
655799118,2cyclictest5819-21ssh09:52:042
655799118,2cyclictest3741-21ssh11:27:122
655799118,2cyclictest3255-21grep08:10:152
655799118,2cyclictest29188-21ssh11:42:212
655799118,2cyclictest23577-21tr07:45:152
655799118,2cyclictest19643-21sh10:20:032
655799117,3cyclictest15874-21head08:35:172
655399119,1cyclictest17104-21grep07:30:181
655399118,2cyclictest32763-21turbostat.cron08:05:011
655399118,2cyclictest3204-21ssh11:45:491
655399118,2cyclictest29009-21chrt09:03:101
655399118,2cyclictest28234-21ssh09:45:441
655399118,2cyclictest26545-21cut07:50:171
655399118,2cyclictest25843-21ssh11:01:501
655399118,2cyclictest25206-21grep07:45:281
655399118,2cyclictest25174-21ssh11:20:251
655399118,2cyclictest17291-21ssh11:55:051
655399118,2cyclictest15627-21tail07:25:271
655399118,2cyclictest14184-21ssh11:33:461
655399117,3cyclictest2695-21taskset08:09:581
655399115,4cyclictest24716-21aten2_r7power_e08:55:151
655399111,10cyclictest0-21swapper/110:00:511
655199118,2cyclictest981-21tr11:45:150
655199118,2cyclictest9062-21grep08:20:200
655199118,2cyclictest7059-21ssh10:12:000
655199118,2cyclictest30242-21head10:25:260
655199118,2cyclictest29401-21ssh10:05:400
655199118,2cyclictest21808-21cut07:40:170
655199118,2cyclictest13586-21ssh12:11:290
655199118,2cyclictest13371-21cut10:35:170
655199117,3cyclictest20690-21grep08:45:180
655199116,4cyclictest516-21gdbus11:25:230
655199111,7cyclictest18940-21cstates12:15:140
655999108,1cyclictest26434-21sed07:50:153
655999108,1cyclictest25988-21mailstats08:55:233
655999108,1cyclictest21128-21apache_accesses07:40:123
655999108,1cyclictest15150-21apt08:35:143
655999107,2cyclictest7060-21expr07:10:143
655999107,2cyclictest5767-21cat08:15:163
655999107,2cyclictest4823-21taskset08:11:483
655999107,2cyclictest27497-21cat09:00:153
655999107,2cyclictest18969-21tr07:35:153
655999100,9cyclictest2804-21cat08:10:003
655799108,1cyclictest27536-21sh09:00:162
655799108,1cyclictest23038-21expr08:50:172
655799108,1cyclictest23011-21taskset07:42:112
655799108,1cyclictest20116-21cat08:45:152
655799108,1cyclictest16833-21expr07:30:172
655799108,1cyclictest12391-21ssh10:35:032
655799108,1cyclictest11923-21cat07:20:162
655799107,2cyclictest719-21cut08:05:152
655799107,2cyclictest26351-21sed08:55:272
655799107,2cyclictest18697-21sh08:40:192
655399108,1cyclictest22995-21head08:50:161
655399108,1cyclictest12943-21sendmail_mailst07:20:231
655199108,1cyclictest9581-21expr07:15:150
655199108,1cyclictest293852sleep009:05:010
655199108,1cyclictest26853-21cat07:50:200
655199108,1cyclictest23833-21expr07:45:150
655199108,1cyclictest17720-21sendmail_mailtr07:30:230
655199108,1cyclictest13133-21expr08:30:160
655199107,2cyclictest6686-21awk08:15:230
655199107,2cyclictest30417-21sed08:00:000
655199107,2cyclictest27995-21cut09:00:190
655199107,2cyclictest1832-21grep08:05:230
655199100,9cyclictest24230-21taskset08:53:010
65539990,4cyclictest11791-21unixbench-2d08:25:271
65519996,2cyclictest7051-21tr07:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional