You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 10:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 04, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
989829682,9sleep00-21swapper/019:08:500
983029065,9sleep20-21swapper/219:07:562
967028864,13sleep10-21swapper/119:05:541
989128673,9sleep30-21swapper/319:08:443
275482490,0sleep10-21swapper/121:24:371
1006799320,21cyclictest0-21swapper/122:12:261
1006799310,3cyclictest15732-21sed00:10:251
1007299290,28cyclictest0-21swapper/223:54:152
1007299270,26cyclictest0-21swapper/221:15:322
1007299270,23cyclictest0-21swapper/200:35:082
1007299260,25cyclictest0-21swapper/222:36:292
1007299250,25cyclictest0-21swapper/221:26:532
1007299250,24cyclictest0-21swapper/223:11:592
1007799241,22cyclictest26178-21kworker/3:100:26:333
1007299240,24cyclictest0-21swapper/222:43:572
1007299240,23cyclictest0-21swapper/223:00:512
1007299230,22cyclictest0-21swapper/200:20:152
1007299230,21cyclictest0-21swapper/221:00:122
1007299230,1cyclictest0-21swapper/221:37:142
1006799230,22cyclictest0-21swapper/123:49:401
1006699230,17cyclictest0-21swapper/000:03:390
1006699220,1cyclictest0-21swapper/023:19:490
10042220,0sleep20-21swapper/222:06:102
320782210,0chrt0-21swapper/323:22:493
1007299210,17cyclictest0-21swapper/223:24:072
10077992015,2cyclictest39-21ksoftirqd/323:11:443
10072992019,1cyclictest0-21swapper/223:55:362
10072992019,1cyclictest0-21swapper/223:30:582
1006699200,20cyclictest0-21swapper/021:15:320
80952190,0sleep30-21swapper/322:30:093
10072991918,1cyclictest0-21swapper/222:24:302
10072991915,2cyclictest0-21swapper/223:17:532
1007299190,1cyclictest0-21swapper/222:01:212
1007299190,1cyclictest0-21swapper/222:01:202
1007299190,18cyclictest0-21swapper/221:46:072
1007299190,16cyclictest0-21swapper/223:27:202
1006799190,1cyclictest3100-21diskmemload00:00:561
1007299180,3cyclictest0-21swapper/222:48:512
1007299180,2cyclictest0-21swapper/223:47:102
1006699186,6cyclictest28377-21munin-run20:55:000
1006699181,9cyclictest1-21systemd00:19:540
1006699180,2cyclictest0-21swapper/022:18:040
1006699180,0cyclictest0-21swapper/000:35:080
1007799170,3cyclictest0-21swapper/321:30:143
1007799170,14cyclictest0-21swapper/300:24:073
10072991716,1cyclictest0-21swapper/222:57:012
1007299170,17cyclictest0-21swapper/223:38:252
10066991716,1cyclictest13-21ksoftirqd/023:33:580
1006699170,16cyclictest0-21swapper/021:37:140
1006699170,13cyclictest0-21swapper/021:00:110
327372160,3chrt25-21ksoftirqd/123:04:081
1007799160,16cyclictest0-21swapper/321:45:543
10072991614,1cyclictest9371-21ssh22:50:162
10072991611,4cyclictest31093-21kworker/u8:123:44:022
1007299160,12cyclictest0-21swapper/221:56:082
1006799160,1cyclictest0-21swapper/122:22:061
1006799160,14cyclictest0-21swapper/123:18:391
1006699166,7cyclictest9319-21ssh22:50:140
1006699160,11cyclictest0-21swapper/023:56:250
1007799156,3cyclictest225-21systemd-journal19:55:263
10072991511,3cyclictest32-21ksoftirqd/223:05:042
1007299150,15cyclictest0-21swapper/200:33:232
1007299150,13cyclictest0-21swapper/222:34:022
10067991513,2cyclictest0-21swapper/119:40:201
1006799151,14cyclictest3100-21diskmemload00:16:591
1006799150,1cyclictest0-21swapper/100:36:561
1006799150,15cyclictest0-21swapper/121:38:121
10066991514,1cyclictest0-21swapper/021:54:420
1006699151,1cyclictest141rcu_preempt23:05:270
10066991510,1cyclictest0-21swapper/022:24:010
1006699150,1cyclictest0-21swapper/021:46:350
1006699150,1cyclictest0-21swapper/000:24:540
1007799142,11cyclictest0-21swapper/300:00:213
10077991413,1cyclictest0-21swapper/320:21:493
10077991412,1cyclictest0-21swapper/323:05:003
10077991410,2cyclictest2939-21grep22:45:303
10077991410,2cyclictest26528-21tr20:50:163
1007299148,5cyclictest3100-21diskmemload22:17:332
1007299140,3cyclictest0-21swapper/221:11:122
1007299140,2cyclictest0-21swapper/200:25:462
1006799148,2cyclictest23118-21cut19:35:171
1006799142,10cyclictest0-21swapper/121:25:271
10067991412,1cyclictest12513-21sh22:51:241
10067991411,2cyclictest8412-21grep00:25:271
1006799141,12cyclictest21717-21fschecks_time23:55:181
10067991410,3cyclictest16213-21perf20:30:001
1006699147,2cyclictest225-21systemd-journal00:05:020
1006699144,8cyclictest823-21proc_pri19:55:220
1006699144,5cyclictest0-21swapper/021:55:240
1006699142,10cyclictest21140-21ssh21:40:140
1006699141,6cyclictest0-21swapper/022:10:140
10066991412,2cyclictest13-21ksoftirqd/022:31:050
1007799139,3cyclictest8627-21sed22:10:273
1007799139,3cyclictest22025-21sed21:20:283
1007799138,4cyclictest1051-21wc22:25:223
10077991310,2cyclictest30960-21sh23:41:093
10077991310,2cyclictest28301-21wc19:45:343
10077991310,2cyclictest2650-21sed21:05:263
10077991310,2cyclictest23101-21tail20:40:263
10077991310,2cyclictest17122-21ls22:55:003
10077991310,2cyclictest13766-21sed21:15:283
1007799130,12cyclictest21667-21sed19:30:253
1007299139,3cyclictest4083-21cron20:05:002
1007299139,1cyclictest32-21ksoftirqd/222:14:462
1007299138,4cyclictest22289-21cut19:35:002
1007299131,11cyclictest55550irq/126-enp2s0-21:32:212
10072991310,2cyclictest22243-21ssh00:15:152
1006799138,4cyclictest32543-21ssh23:23:111
1006799138,4cyclictest18331-21ssh21:19:011
1006799137,6cyclictest487-21avahi-daemon22:31:321
1006799131,1cyclictest0-21swapper/121:45:271
10067991310,2cyclictest832-21wc19:55:231
10067991310,2cyclictest30712-21grep00:20:161
10067991310,2cyclictest26423-21ssh22:40:221
10067991310,2cyclictest11524-21wc20:20:011
1006799130,1cyclictest0-21swapper/123:40:121
1006799130,12cyclictest3100-21diskmemload21:10:021
1006699135,4cyclictest10312-21sh23:48:480
10066991310,2cyclictest20960-21ls20:40:010
10066991310,2cyclictest15524-21ssh00:30:160
1006699130,1cyclictest0-21swapper/022:02:520
1006699130,1cyclictest0-21swapper/022:02:520
1007799129,2cyclictest5855-21rm23:45:273
1007799129,2cyclictest4458-21cut20:05:133
1007799129,2cyclictest4458-21cut20:05:133
1007799129,2cyclictest22998-21grep00:15:193
1007799129,2cyclictest22852-21wc21:40:263
1007799129,2cyclictest2048-21grep20:00:153
1007799129,2cyclictest20411-21cron23:55:003
1007799129,2cyclictest19029-21tr22:55:213
1007799129,2cyclictest15672-21cut22:15:173
1007799129,2cyclictest14513-21sed19:15:283
1007799129,2cyclictest12007-21sort23:30:173
1007799129,2cyclictest10106-21sed20:15:183
1007799128,3cyclictest4849-21cut21:10:203
1007799128,3cyclictest19293-21grep19:25:253
1007799122,4cyclictest452-21systemd-logind00:05:023
10077991211,1cyclictest26178-21kworker/3:100:30:533
10077991210,1cyclictest8851-21ssh21:51:353
1007799120,5cyclictest0-21swapper/300:13:303
1007799120,11cyclictest4064-21uname23:25:203
1007799120,11cyclictest21576-21chrt23:16:123
1007799120,11cyclictest0-21swapper/321:57:013
1007299129,2cyclictest7172-21tr20:10:122
1007299129,2cyclictest6068-21sort00:05:172
1007299129,2cyclictest6054-21head20:05:262
1007299129,2cyclictest6054-21head20:05:252
1007299129,2cyclictest25532-21grep20:45:272
1007299129,2cyclictest25011-21basename19:40:162
1007299129,2cyclictest20948-21sh20:40:002
1007299129,2cyclictest18322-21sort19:25:162
1007299128,3cyclictest25514-21sh21:23:022
1007299127,1cyclictest225-21systemd-journal20:53:002
10072991210,1cyclictest27818-21df19:45:302
10072991210,1cyclictest0-21swapper/221:40:052
1006799129,2cyclictest30846-21head19:50:211
1006799129,2cyclictest27980-21sed20:50:281
1006799129,2cyclictest23838-21cat22:00:261
1006799129,2cyclictest23838-21cat22:00:261
1006799129,2cyclictest20280-21chrt23:35:161
1006799129,2cyclictest16957-21grep20:30:151
1006799129,2cyclictest13623-21head19:15:181
1006799121,11cyclictest3100-21diskmemload22:17:271
10067991211,1cyclictest3100-21diskmemload21:44:541
1006799120,12cyclictest0-21swapper/120:49:571
1006799120,11cyclictest1288-21grep22:45:181
1006699129,2cyclictest9068-21basename20:15:000
1006699129,2cyclictest6729-21date20:10:010
1006699129,2cyclictest6729-21date20:10:010
1006699129,2cyclictest671-21grep22:25:190
1006699129,2cyclictest26117-21taskset20:50:000
1006699129,2cyclictest21916-21cat21:20:280
1006699129,2cyclictest14503-21grep23:50:260
1006699129,2cyclictest10694-21ssh21:14:250
1006699128,2cyclictest32209-21ssh23:42:080
10066991211,1cyclictest0-21swapper/022:44:560
10066991210,1cyclictest28937-21sed23:20:260
1006699120,12cyclictest0-21swapper/023:00:500
1006699120,11cyclictest1120-21cut22:45:170
1006699120,10cyclictest0-21swapper/022:36:160
1007799118,2cyclictest31248-21tr21:00:143
1007799118,2cyclictest30840-21tr22:05:173
1007799118,2cyclictest30641-21taskset20:58:523
1007799118,2cyclictest29860-21grep19:50:133
1007799118,2cyclictest25878-21head22:40:173
1007799118,2cyclictest25018-21wget19:40:163
1007799118,2cyclictest24146-21ssh23:56:233
1007799118,2cyclictest23868-21wc19:35:253
1007799118,2cyclictest23865-21grep20:45:143
1007799118,2cyclictest23553-21ssh22:20:103
1007799118,2cyclictest22422-21grep22:00:153
1007799118,2cyclictest22422-21grep22:00:153
1007799118,2cyclictest20174-21ssh22:36:413
1007799118,2cyclictest15554-21tr19:20:163
1007799118,2cyclictest15245-21ntpq20:25:203
1007799118,2cyclictest12828-21ssh21:35:143
1007799118,2cyclictest1178-21sh21:27:503
1007799118,1cyclictest10544-21cut19:10:133
1007799111,9cyclictest0-21swapper/323:06:083
10077991110,1cyclictest26178-21kworker/3:123:36:033
1007299119,1cyclictest14860-21sed20:25:162
1007299119,1cyclictest14517-21sed19:15:292
1007299119,1cyclictest13898-21sh00:10:122
1007299118,2cyclictest9423-21grep20:15:152
1007299118,2cyclictest29641-21cut19:50:132
1007299118,2cyclictest29333-21cut00:00:142
1007299118,2cyclictest17066-21cut20:30:162
1007299118,2cyclictest1358-21find22:25:242
1007299118,2cyclictest13346-21sed20:20:272
1007299118,2cyclictest1194-21sed19:55:272
1007299118,2cyclictest10101-21sh21:52:332
1007299110,10cyclictest234692sleep220:44:002
1006799119,1cyclictest3297-21grep23:25:161
1006799118,2cyclictest9475-21ssh21:52:041
1006799118,2cyclictest6544-21cat00:05:201
1006799118,2cyclictest3673-21grep20:00:281
1006799118,2cyclictest25-21ksoftirqd/122:56:511
1006799118,2cyclictest21673-21ssh00:34:101
1006799118,2cyclictest21220-21ssh22:37:311
1006799118,2cyclictest20094-21ssh23:54:461
1006799118,2cyclictest12959-21cat23:30:261
1006799118,2cyclictest12267-21grep23:10:271
1006799118,2cyclictest11963-21tr20:20:161
1006799118,2cyclictest1007-21wget21:05:141
1006799115,5cyclictest32291-21ssh22:25:151
1006799115,2cyclictest225-21systemd-journal20:40:011
1006799111,1cyclictest0-21swapper/119:45:291
1006799110,10cyclictest30441-21ssh22:05:111
1006699119,1cyclictest4758-21grep21:30:150
1006699119,1cyclictest30249-21grep19:50:160
1006699119,1cyclictest245182sleep019:39:110
1006699119,1cyclictest18509-21cat19:25:170
1006699119,1cyclictest16108-21taskset20:28:570
1006699119,1cyclictest15448-21grep19:20:160
1006699118,2cyclictest3172-21tr23:25:150
1006699118,2cyclictest3100-21diskmemload00:10:200
1006699118,2cyclictest22073-21grep20:40:160
1006699118,2cyclictest19579-21cat23:35:120
1006699118,2cyclictest17091-21cut20:30:160
1006699118,2cyclictest10012-21ssh23:10:020
1006699116,2cyclictest225-21systemd-journal20:20:000
1006699116,1cyclictest6087-21latency_hist22:10:010
1006699115,5cyclictest3719-21vmstat20:00:290
1006699115,2cyclictest475-21in:imuxsock19:48:540
1006699113,7cyclictest942-21/usr/sbin/munin21:05:120
10066991110,1cyclictest0-21swapper/021:28:200
1006699110,10cyclictest0-21swapper/000:25:110
1006699110,0cyclictest0-21swapper/022:57:110
1007799107,2cyclictest8579-21awk20:10:243
1007799107,2cyclictest19579-21tail20:35:163
1007799107,2cyclictest17603-21ntpq20:30:203
1007799104,3cyclictest23222-21apache_volume00:35:113
1007299108,1cyclictest29712sleep221:08:362
1007299107,2cyclictest28686-21cut20:55:142
1007299107,2cyclictest23004-21chrt19:35:162
1007299107,2cyclictest17520-21sort19:25:002
1007299107,2cyclictest11685-21taskset19:10:212
1006799108,1cyclictest88222sleep120:12:131
1006799108,1cyclictest62962sleep120:08:011
1006799108,1cyclictest62962sleep120:08:011
1006799108,1cyclictest4972sleep121:02:421
1006799108,1cyclictest4201-21ssh21:30:021
1006799108,1cyclictest232292sleep120:40:541
1006799108,1cyclictest16061-21grep19:20:191
1006799108,1cyclictest15708-21grep21:55:251
1006799107,2cyclictest3994-21rm23:05:281
1006799107,2cyclictest29894-21wc20:55:221
1006799107,2cyclictest19637-21chrt19:28:181
1006799107,2cyclictest12471-21taskset19:13:061
1006799104,1cyclictest452-21systemd-logind19:34:041
1006699109,1cyclictest12484-21taskset19:13:160
1006699108,1cyclictest28595-21taskset20:55:070
1006699108,1cyclictest220192sleep019:33:210
1006699107,2cyclictest15105-21ls19:20:000
100669998,1cyclictest25547-21tune2fs19:40:180
100669998,1cyclictest12196-21expr20:20:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional