You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 15:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Apr 17, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5926210086,9sleep00-21swapper/007:05:050
795128875,9sleep20-21swapper/207:08:362
660728264,13sleep10-21swapper/107:05:181
800827757,16sleep30-21swapper/307:09:213
85242620,0sleep30-21swapper/309:33:303
8149992523,1cyclictest0-21swapper/211:36:262
814999250,24cyclictest0-21swapper/211:23:322
814299250,24cyclictest0-21swapper/009:13:400
814999240,24cyclictest0-21swapper/211:17:122
814799242,5cyclictest28464-21latency_hist09:00:011
814299240,0cyclictest0-21swapper/011:07:530
8142992222,0cyclictest0-21swapper/010:13:000
223482220,0sleep30-21swapper/310:39:583
814299210,1cyclictest0-21swapper/009:33:580
814299210,19cyclictest0-21swapper/012:36:050
815299200,19cyclictest0-21swapper/310:16:343
815299200,19cyclictest0-21swapper/310:16:343
8147992019,1cyclictest0-21swapper/111:46:531
814799200,1cyclictest0-21swapper/108:00:221
814299203,10cyclictest225-21systemd-journal08:05:000
8142992019,1cyclictest25714-21apache211:42:170
815299190,4cyclictest0-21swapper/310:30:523
814799191,1cyclictest0-21swapper/112:05:131
814799191,1cyclictest0-21swapper/112:05:121
814799190,1cyclictest141rcu_preempt08:55:001
814799190,1cyclictest0-21swapper/110:57:301
8142991919,0cyclictest0-21swapper/011:16:010
814299190,19cyclictest0-21swapper/011:25:550
814299190,0cyclictest0-21swapper/012:16:070
815299181,13cyclictest0-21swapper/310:40:163
815299180,18cyclictest0-21swapper/309:59:413
814999180,17cyclictest0-21swapper/208:23:392
8147991814,4cyclictest0-21swapper/111:22:111
815299171,12cyclictest18196-21/usr/sbin/munin09:20:183
814999173,5cyclictest20017-21idleruntime-cro10:00:002
814799174,2cyclictest141rcu_preempt12:38:061
8147991716,1cyclictest0-21swapper/109:50:191
814799170,3cyclictest0-21swapper/109:13:471
814799170,16cyclictest0-21swapper/111:15:361
8142991714,2cyclictest13-21ksoftirqd/009:49:420
8142991713,4cyclictest25714-21apache209:56:310
815299166,6cyclictest537-21snmpd11:16:263
8152991612,3cyclictest29232-21cron12:39:593
815299160,16cyclictest0-21swapper/312:13:143
8149991614,2cyclictest979-21diskmemload12:07:122
8149991614,2cyclictest979-21diskmemload12:07:112
814999160,12cyclictest0-21swapper/212:25:482
8147991616,0cyclictest0-21swapper/109:15:401
8147991611,3cyclictest38150irq/131-snd_hda11:55:091
814799160,11cyclictest0-21swapper/107:45:111
814299160,2cyclictest0-21swapper/011:49:460
8152991511,3cyclictest13398-21sed08:25:263
8152991511,3cyclictest13398-21sed08:25:263
814999150,15cyclictest0-21swapper/211:10:132
814999150,15cyclictest0-21swapper/209:29:542
814999150,14cyclictest0-21swapper/210:00:432
814999150,13cyclictest0-21swapper/212:37:322
814799153,12cyclictest979-21diskmemload09:31:161
814799150,14cyclictest0-21swapper/109:48:531
814799150,13cyclictest31006-21ssh12:20:441
814299151,14cyclictest0-21swapper/009:19:310
814299150,15cyclictest0-21swapper/010:53:340
815299148,5cyclictest11147-21ssh12:09:513
815299148,5cyclictest11147-21ssh12:09:503
8152991411,2cyclictest405-21sort08:00:163
8152991410,3cyclictest4999-21head09:50:193
8152991410,2cyclictest21929-21tail07:35:253
815299140,1cyclictest0-21swapper/311:20:133
815299140,13cyclictest0-21swapper/309:27:483
814999147,3cyclictest0-21swapper/211:55:132
8149991411,2cyclictest0-21swapper/210:35:002
8149991410,3cyclictest9409-21ssh09:14:592
8149991410,3cyclictest31851-21df10:45:132
8149991410,1cyclictest0-21swapper/211:47:242
814999140,14cyclictest0-21swapper/211:54:422
8147991412,1cyclictest979-21diskmemload10:44:071
814799141,12cyclictest8592-21unixbench_multi08:15:261
814799140,1cyclictest0-21swapper/112:29:081
814799140,1cyclictest0-21swapper/110:53:421
8142991411,3cyclictest13-21ksoftirqd/011:00:050
8142991410,3cyclictest24675-21cat10:40:250
8142991410,2cyclictest4678-21grep08:10:140
8142991410,2cyclictest0-21swapper/010:59:510
814299140,13cyclictest0-21swapper/009:20:130
107782140,0chrt0-21swapper/209:15:172
815299139,3cyclictest3302-21grep11:25:273
815299139,3cyclictest23550-21sh10:59:593
815299139,3cyclictest19894-21wc08:40:193
815299138,3cyclictest8402-21tr10:50:203
8152991312,1cyclictest0-21swapper/312:24:543
8152991310,2cyclictest20023-21grep11:35:263
815299130,11cyclictest0-21swapper/311:32:433
814999139,3cyclictest15573-21basename07:25:152
8149991311,1cyclictest0-21swapper/210:29:172
8149991310,2cyclictest8181-21grep08:15:212
8149991310,2cyclictest4839-21grep08:10:152
8149991310,2cyclictest28921-21basename09:00:122
8149991310,2cyclictest26082-21cat07:45:172
8149991310,2cyclictest17591-21cron11:34:592
814799139,3cyclictest12914-21grep09:35:251
814799139,2cyclictest13657-21cat12:10:251
814799135,4cyclictest1553-21ssh09:29:341
8147991310,2cyclictest31394-21grep09:05:141
8147991310,2cyclictest22739-21tr07:40:141
8147991310,2cyclictest1728-21grep11:05:211
8147991310,2cyclictest12474-21tr09:55:171
8147991310,2cyclictest12038-21cut11:50:191
814799130,9cyclictest0-21swapper/108:30:141
814799130,1cyclictest15064-21df_abs10:35:131
814799130,1cyclictest0-21swapper/110:46:081
814299139,3cyclictest28243-21sort12:00:150
814299139,3cyclictest19469-21wc07:30:240
814299139,2cyclictest22864-21sed08:45:260
814299139,2cyclictest15808-21grep08:30:260
814299136,4cyclictest537-21snmpd11:50:060
814299132,8cyclictest13443-21aten2_r7power_p10:15:140
814299132,8cyclictest13443-21aten2_r7power_p10:15:130
814299131,11cyclictest5076-21sensors12:05:230
814299131,11cyclictest5076-21sensors12:05:230
8142991310,2cyclictest4430-21cron10:10:000
8142991310,2cyclictest29582-21latency_hist07:55:000
8142991310,2cyclictest21014-21cat12:35:010
8142991310,2cyclictest15244-21ssh10:35:140
8142991310,2cyclictest13332-21sed08:25:260
8142991310,2cyclictest13332-21sed08:25:260
815299129,2cyclictest9684-21grep07:10:223
815299129,2cyclictest3218-21sort11:45:163
815299129,2cyclictest31898-21sh10:45:143
815299129,2cyclictest29666-21sed09:00:163
815299129,2cyclictest28437-21readlink09:00:013
815299129,2cyclictest28120-21cut07:50:173
815299129,2cyclictest25308-21cat11:00:193
815299129,2cyclictest24828-21tr08:50:193
815299129,2cyclictest21221-21sed09:40:253
815299129,2cyclictest20148-21ssh12:15:113
815299129,2cyclictest17229-21sed08:35:173
815299129,2cyclictest15250-21cron07:25:003
815299129,2cyclictest13792-21grep12:30:183
815299129,2cyclictest10515-21sed11:10:253
815299128,3cyclictest23627-21tail10:20:253
815299128,2cyclictest438-21grep09:05:263
815299127,4cyclictest5507-21ssh12:25:183
815299120,8cyclictest27861-21grep12:00:133
814999129,2cyclictest3264-21sleep09:49:592
814999129,2cyclictest30003-21ls12:20:212
814999129,2cyclictest27972-21sed08:55:282
814999129,2cyclictest27718-21grep07:50:152
814999129,2cyclictest24358-21wc10:40:212
814999129,2cyclictest23354-21dump-pmu-power08:50:012
814999129,2cyclictest22540-21cat10:40:012
814999129,2cyclictest20895-21grep07:35:162
814999129,2cyclictest18937-21tr08:40:142
814999129,2cyclictest12375-21sort08:25:162
814999129,2cyclictest12375-21sort08:25:152
814999128,3cyclictest26136-21cron09:25:002
814999128,3cyclictest23444-21sed07:40:172
814999128,3cyclictest13959-21if_enp2s012:30:192
814999127,4cyclictest0-21swapper/212:12:142
814799129,2cyclictest9389-21tr08:20:131
814799129,2cyclictest9100-21grep07:10:161
814799129,2cyclictest5614-21grep10:10:151
814799129,2cyclictest28095-21apt12:20:011
814799129,2cyclictest19041-21ssh09:20:171
814799129,2cyclictest17207-21head08:35:181
814799129,2cyclictest14800-21grep07:20:241
814799129,2cyclictest14517-21cat10:15:191
814799129,2cyclictest14517-21cat10:15:191
814799128,3cyclictest3900-21ssh11:25:491
814799128,3cyclictest29031-21cut10:05:131
814799128,3cyclictest20937-21tune2fs07:35:161
814799122,2cyclictest225-21systemd-journal11:34:591
814799120,1cyclictest0-21swapper/108:45:121
814299129,2cyclictest5668-21munin-run10:30:010
814299129,2cyclictest29979-21basename07:55:140
814299129,2cyclictest26035-21sh11:40:000
814299129,2cyclictest22295-21ssh10:00:240
814299129,2cyclictest20517-21grep07:35:140
814299129,2cyclictest10967-21grep08:20:250
814299124,4cyclictest537-21snmpd09:52:430
814299120,11cyclictest29510-21sort09:00:160
815299118,2cyclictest9595-21tr08:20:153
815299118,2cyclictest9108-21basename08:20:003
815299118,2cyclictest4258-21ssh09:11:003
815299118,2cyclictest4197-21taskset08:08:433
815299118,2cyclictest3614-21ssh11:06:243
815299118,2cyclictest30454-21cut10:25:143
815299118,2cyclictest29686-21ssh11:41:173
815299118,2cyclictest28162-21wc09:45:143
815299118,2cyclictest25740-21cut07:45:143
815299118,2cyclictest22913-21grep07:40:143
815299118,2cyclictest18953-21ssh11:55:053
815299118,2cyclictest17423-21chrt07:27:283
815299118,2cyclictest1570-21ssh10:07:473
815299118,2cyclictest14472-21expr08:30:153
815299118,2cyclictest11113-21cut09:35:143
815299118,2cyclictest10577-21ssh11:50:043
815299118,2cyclictest10307-21ssh10:13:033
815299117,3cyclictest21124-21ssh10:00:143
815299117,3cyclictest18170-21tr07:30:123
815299110,10cyclictest9631-21ssh09:15:023
814999118,2cyclictest7930-21ssh09:33:022
814999118,2cyclictest7438-21ssh10:10:502
814999118,2cyclictest666-21cat11:05:132
814999118,2cyclictest5000-21cut09:50:192
814999118,2cyclictest32464-21tr08:00:142
814999118,2cyclictest31813-21taskset07:58:212
814999118,2cyclictest31604-21expr09:05:142
814999118,2cyclictest31292-21ssh11:04:262
814999118,2cyclictest30848-21sh12:01:252
814999118,2cyclictest2448-21uname11:25:192
814999118,2cyclictest19980-21ssh12:15:042
814999118,2cyclictest19772-21rm10:18:542
814999118,2cyclictest19772-21rm10:18:542
814999118,2cyclictest18263-21taskset08:36:052
814999118,2cyclictest12976-21ssh09:35:262
814999118,2cyclictest10500-21perf07:15:002
814999113,4cyclictest225-21systemd-journal10:54:592
814999111,6cyclictest225-21systemd-journal09:45:002
814799119,1cyclictest13396-21grep08:25:261
814799119,1cyclictest13396-21grep08:25:251
814799118,2cyclictest7158-21if_enp2s010:30:181
814799118,2cyclictest38150irq/131-snd_hda09:41:001
814799118,2cyclictest2983-21ssh10:27:551
814799118,2cyclictest29178-21rm11:40:541
814799118,2cyclictest26688-21ssh10:22:391
814799118,2cyclictest20302-21rm11:35:351
814799118,2cyclictest17738-21ssh12:32:361
814799118,2cyclictest16449-21uname07:25:201
814799118,2cyclictest14306-21rm11:13:181
814799117,3cyclictest4307-21idleruntime-cro08:09:591
814799114,5cyclictest29617-21grep09:00:171
814799114,4cyclictest28308-21fschecks_time12:00:151
814799110,10cyclictest26190-21ssh11:00:281
814299118,2cyclictest8911-21grep11:10:130
814299118,2cyclictest6722-21ssh12:25:310
814299118,2cyclictest393-21ssh12:22:100
814299118,2cyclictest31247-21ssh10:45:080
814299118,2cyclictest29579-21ssh11:22:060
814299118,2cyclictest27305-21sed07:49:590
814299118,2cyclictest26846-21cut09:25:140
814299118,2cyclictest22784-21cut10:20:170
814299118,2cyclictest16946-21wc07:25:270
814299118,2cyclictest16633-21ssh12:12:370
814299118,2cyclictest14789-21ssh09:36:520
814299118,2cyclictest13394-21cut07:20:130
814299118,2cyclictest10003-21grep11:30:140
814299115,5cyclictest6181-21apt-get10:30:130
814299110,1cyclictest0-21swapper/009:40:250
814299110,10cyclictest9309-21grep07:10:170
815299108,1cyclictest29899-21cut07:55:153
815299107,2cyclictest6676-21cron08:15:003
815299107,2cyclictest10815-21cut07:15:143
815299106,3cyclictest21516-21grep08:45:143
814999108,1cyclictest7236-21sh10:50:102
814999108,1cyclictest3967-21chrt08:05:442
814999108,1cyclictest26346-21chrt11:40:032
814999108,1cyclictest24585-21cat08:50:162
814999108,1cyclictest22184-21ls10:20:142
814999108,1cyclictest19928-21chrt07:34:072
814999108,1cyclictest131132sleep207:20:062
814999108,1cyclictest125412sleep207:16:012
814999107,2cyclictest14572-21cat08:30:152
814999100,9cyclictest0-21swapper/210:05:132
814799108,1cyclictest29509-21chrt07:54:141
814799107,2cyclictest30648-21hwlatdetect07:55:191
814799107,2cyclictest22652-21ssh10:00:301
814799107,2cyclictest10966-21cat07:15:151
814799100,9cyclictest5335-21expr08:10:171
814799100,9cyclictest207372sleep108:42:071
814299108,1cyclictest760-21chrt09:07:300
814299108,1cyclictest24651-21chrt07:43:320
814299107,2cyclictest7598-21sort08:15:170
814299107,2cyclictest21553-21ssh11:55:440
814299107,2cyclictest20929-21chrt08:44:330
814299103,3cyclictest225-21systemd-journal08:50:000
814299100,9cyclictest4022-21taskset08:06:280
814299100,9cyclictest12825-21taskset07:19:390
814299100,5cyclictest0-21swapper/008:55:150
81479996,2cyclictest18288-21cat07:30:141
81429997,1cyclictest17927-21sed08:35:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional