You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 15:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Jul 26, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
640829885,9sleep00-21swapper/007:09:070
628629278,9sleep20-21swapper/207:07:322
645828961,9sleep30-21swapper/307:09:463
625228564,6sleep10-21swapper/107:07:061
312912630,0sleep30-21swapper/309:47:403
257272240,0sleep30-21swapper/310:23:113
656799233,18cyclictest0-21swapper/208:30:162
656199230,2cyclictest0-21swapper/010:04:090
657199220,21cyclictest0-21swapper/311:26:063
656799220,21cyclictest0-21swapper/210:59:072
6567992114,6cyclictest0-21swapper/212:15:322
656799210,20cyclictest0-21swapper/212:07:392
657199200,1cyclictest0-21swapper/310:00:033
6567992019,1cyclictest0-21swapper/211:04:432
6567992018,1cyclictest11795-21/usr/sbin/munin10:15:152
656799200,20cyclictest0-21swapper/209:55:482
656799200,20cyclictest0-21swapper/209:55:472
656599201,3cyclictest1292-21apache_accesses09:30:141
656199202,0cyclictest141rcu_preempt12:12:440
6561992019,1cyclictest0-21swapper/011:13:310
657199190,1cyclictest0-21swapper/312:13:443
6567991916,3cyclictest0-21swapper/210:10:232
656799191,18cyclictest0-21swapper/210:42:392
656799190,19cyclictest0-21swapper/212:32:422
656799190,19cyclictest0-21swapper/209:38:512
656799190,0cyclictest0-21swapper/210:46:062
656599193,2cyclictest28362-21idleruntime-cro12:40:001
6565991918,1cyclictest32042-21diskmemload10:34:521
656599190,19cyclictest0-21swapper/109:14:361
656599190,17cyclictest0-21swapper/109:23:111
6571991815,2cyclictest39-21ksoftirqd/311:34:373
656599180,18cyclictest0-21swapper/112:10:341
656599180,16cyclictest0-21swapper/109:35:371
657199170,1cyclictest0-21swapper/311:41:593
657199170,17cyclictest0-21swapper/311:24:093
6567991712,5cyclictest0-21swapper/209:17:232
656799170,17cyclictest0-21swapper/210:23:022
6565991713,3cyclictest0-21swapper/112:28:081
6565991710,6cyclictest27650-21apache211:41:391
656599170,16cyclictest0-21swapper/111:59:471
656599170,15cyclictest0-21swapper/111:10:321
656199173,7cyclictest23304-21tr11:20:010
656199170,16cyclictest0-21swapper/011:41:580
657199162,6cyclictest0-21swapper/309:30:003
656799160,16cyclictest0-21swapper/209:28:002
656799160,15cyclictest0-21swapper/211:24:412
656599166,3cyclictest225-21systemd-journal07:34:591
656599162,13cyclictest17503-21ssh12:33:101
656599160,15cyclictest0-21swapper/110:00:051
656599160,0cyclictest0-21swapper/111:26:531
656199160,2cyclictest0-21swapper/010:58:430
657199157,5cyclictest39-21ksoftirqd/311:50:203
6571991513,2cyclictest39-21ksoftirqd/309:22:033
6571991513,1cyclictest0-21swapper/312:27:183
657199150,14cyclictest0-21swapper/312:18:393
656799155,5cyclictest225-21systemd-journal11:50:002
656799151,13cyclictest0-21swapper/209:13:592
656799150,15cyclictest0-21swapper/209:46:252
656799150,15cyclictest0-21swapper/209:33:212
6565991513,1cyclictest21865-21python10:20:251
656199156,2cyclictest141rcu_preempt11:58:180
656199153,7cyclictest9138-21perf07:15:000
6561991512,2cyclictest19753-21sendmail-msp10:20:010
656199151,1cyclictest141rcu_preempt10:22:560
6561991510,1cyclictest13-21ksoftirqd/009:39:340
656199150,2cyclictest0-21swapper/011:24:410
657199148,5cyclictest39-21ksoftirqd/310:39:053
657199147,2cyclictest9403-21ssh11:50:003
6571991412,1cyclictest39-21ksoftirqd/310:13:013
6571991410,3cyclictest6199-21tr10:50:153
6571991410,3cyclictest28500-21grep12:20:173
6571991410,3cyclictest19727-21cat09:40:273
657199140,1cyclictest0-21swapper/309:19:213
656799149,4cyclictest1235-21ntpd12:26:192
656799149,3cyclictest27221-21sleep09:00:002
656799145,3cyclictest18068-21apt-get11:55:132
6567991413,1cyclictest0-21swapper/212:13:062
6567991410,3cyclictest23268-21sh11:20:002
656799140,3cyclictest0-21swapper/207:50:142
656799140,14cyclictest0-21swapper/209:23:302
6565991414,0cyclictest0-21swapper/109:27:371
6565991411,2cyclictest20795-21munin-run10:40:001
6565991410,3cyclictest6670-21ssh09:52:131
656599140,3cyclictest0-21swapper/110:26:401
6561991412,2cyclictest13-21ksoftirqd/012:30:480
6561991411,2cyclictest28150-21grep09:00:160
6561991410,3cyclictest28363-21basename12:40:000
6561991410,3cyclictest25609-21tr09:25:160
6561991410,3cyclictest21022-21grep08:45:150
656199140,4cyclictest0-21swapper/012:15:080
656199140,12cyclictest0-21swapper/012:23:320
657199139,3cyclictest28152-21cut12:00:233
657199139,3cyclictest17514-21basename09:40:003
657199139,3cyclictest12414-21ssh12:30:153
657199139,3cyclictest11070-21grep07:15:253
657199137,2cyclictest381rcuc/307:30:003
6571991312,1cyclictest0-21swapper/310:56:423
6571991310,2cyclictest5687-21grep08:15:143
6571991310,2cyclictest3819-21tr12:05:213
6571991310,2cyclictest2690-21cut09:50:153
6571991310,2cyclictest22446-21cron08:49:593
6571991310,2cyclictest17471-21cron08:40:003
657199130,11cyclictest0-21swapper/309:13:493
656799139,3cyclictest32749-21ssh11:25:162
656799139,3cyclictest30759-21cat08:00:012
656799139,3cyclictest20690-21tr12:35:162
656799139,3cyclictest13756-21tail10:35:182
6567991312,1cyclictest0-21swapper/207:15:122
6567991310,3cyclictest0-21swapper/211:41:232
6567991310,2cyclictest6808-21sed08:15:222
6567991310,2cyclictest29040-21grep10:25:192
6567991310,2cyclictest13004-21sed07:20:222
6567991310,2cyclictest12463-21sh08:30:012
656799130,3cyclictest0-21swapper/211:33:302
656799130,13cyclictest0-21swapper/211:52:502
656599139,4cyclictest10550irq/122-ahci[0008:39:591
656599139,3cyclictest20558-21grep12:15:211
656599139,2cyclictest21407-21tr07:40:151
6565991311,1cyclictest32042-21diskmemload12:09:031
656599131,11cyclictest32042-21diskmemload09:44:391
6565991310,2cyclictest540-21cron08:05:001
6565991310,2cyclictest540-21cron08:05:001
6565991310,2cyclictest28237-21awk07:55:001
6565991310,2cyclictest24527-21tune2fs11:20:171
6565991310,2cyclictest10603-21ls07:15:211
656599130,1cyclictest0-21swapper/111:51:461
656599130,1cyclictest0-21swapper/110:40:451
656599130,12cyclictest0-21swapper/111:48:331
6561991311,2cyclictest13-21ksoftirqd/011:25:010
6561991311,1cyclictest13-21ksoftirqd/012:27:590
6561991310,2cyclictest9383-21grep11:30:190
6561991310,2cyclictest4813-21grep10:30:160
6561991310,2cyclictest30662-21grep09:05:150
6561991310,2cyclictest28580-21basename07:55:150
6561991310,2cyclictest24547-21ssh11:40:000
6561991310,2cyclictest17562-21ssh11:55:000
656199130,12cyclictest4507-21grep12:05:270
657199129,2cyclictest55650irq/127-enp2s0-11:56:533
657199129,2cyclictest318-21wc11:05:253
657199129,2cyclictest31659-21grep08:00:173
657199129,2cyclictest31659-21grep08:00:163
657199129,2cyclictest2972-21cron10:10:013
657199129,2cyclictest26717-21cut07:50:163
657199129,2cyclictest24118-21wc11:00:233
657199129,2cyclictest2094-21wc08:05:233
657199128,1cyclictest39-21ksoftirqd/310:33:183
657199124,5cyclictest49-21kcompactd010:45:113
6571991210,2cyclictest39-21ksoftirqd/309:34:083
657199120,1cyclictest0-21swapper/311:36:443
656799129,2cyclictest3308-21grep08:10:142
656799129,2cyclictest19299-21rm09:40:212
656799128,3cyclictest9095-21uname08:20:182
656799128,3cyclictest28170-21cut09:00:172
656799128,3cyclictest23598-21wc07:45:002
656799128,3cyclictest19885-21dpkg08:45:002
656799120,1cyclictest0-21swapper/211:37:102
656799120,12cyclictest0-21swapper/210:33:452
656799120,11cyclictest0-21swapper/208:00:132
656799120,11cyclictest0-21swapper/208:00:132
656599129,2cyclictest9236-21ssh09:15:171
656599129,2cyclictest8772-21head08:20:171
656599129,2cyclictest3050-21wc08:10:011
656599129,2cyclictest29978-21rm12:01:321
656599129,2cyclictest24460-21cut07:45:161
656599129,2cyclictest1888-21basename09:49:591
656599129,2cyclictest18475-21grep08:40:171
656599129,2cyclictest15215-21grep10:55:191
656599129,2cyclictest14845-21apt08:35:011
656599129,2cyclictest11967-21grep09:55:201
656599129,2cyclictest11967-21grep09:55:201
656599127,4cyclictest28704-21sort07:55:161
656599127,2cyclictest30928-21ssh10:06:311
656599127,2cyclictest12217-21cstates10:15:141
656599122,10cyclictest32042-21diskmemload11:04:071
6565991211,1cyclictest0-21swapper/108:50:131
656199129,2cyclictest5377-21grep10:10:260
656199129,2cyclictest28787-21ssh09:45:430
656199129,2cyclictest19149-21grep07:35:150
656199129,2cyclictest15844-21head07:25:270
656199129,2cyclictest13843-21cut08:30:220
656199129,2cyclictest13049-21grep10:35:150
656199128,3cyclictest12459-21cron08:30:010
656199124,7cyclictest577-21idleruntime-cro08:05:000
656199124,7cyclictest577-21idleruntime-cro08:05:000
656199122,1cyclictest141rcu_preempt09:59:210
656199122,1cyclictest141rcu_preempt09:59:210
656199121,0cyclictest141rcu_preempt09:32:200
656199120,0cyclictest141rcu_preempt09:15:320
657199119,1cyclictest17316-21grep07:30:153
657199118,2cyclictest8377-21rm11:10:233
657199118,2cyclictest30690-21head09:05:153
657199118,2cyclictest29372-21ssh10:25:213
657199118,2cyclictest26777-21sed08:55:313
657199118,2cyclictest24442-21sed08:50:263
657199118,2cyclictest22378-21cat07:40:203
657199118,2cyclictest22103-21wc12:35:273
657199118,2cyclictest21756-21expr10:40:163
657199118,2cyclictest19038-21cut07:35:143
657199118,2cyclictest15216-21ssh11:15:053
657199118,2cyclictest12028-21tr07:20:153
657199118,2cyclictest10588-21ssh09:55:083
657199118,2cyclictest10588-21ssh09:55:073
657199117,3cyclictest24446-21cut07:45:163
657199115,4cyclictest39-21ksoftirqd/308:20:123
657199110,10cyclictest11893-21ssh10:15:143
656799119,1cyclictest27830-21grep12:00:212
656799118,2cyclictest8804-21ssh09:53:542
656799118,2cyclictest8436-21awk11:10:242
656799118,2cyclictest7146-21grep07:10:142
656799118,2cyclictest32059-21basename09:10:002
656799118,2cyclictest30297-21ssh12:20:532
656799118,2cyclictest29666-21ssh10:05:312
656799118,2cyclictest2407-21ssh11:06:472
656799118,2cyclictest23484-21sed08:50:172
656799118,2cyclictest22168-21taskset08:46:272
656799118,2cyclictest19224-21cut10:00:152
656799112,7cyclictest5951-21ssh10:50:132
656599118,2cyclictest8634-21sed07:10:261
656599118,2cyclictest6047-21ssh10:10:521
656599118,2cyclictest5688-21sh10:50:021
656599118,2cyclictest31797-21ssh10:45:281
656599118,2cyclictest2775-21ssh11:07:031
656599118,2cyclictest27549-21ssh12:20:091
656599118,2cyclictest22854-21sh11:19:411
656599118,2cyclictest21060-21sendmail-msp07:40:001
656599118,2cyclictest17005-21cut11:35:161
656599118,2cyclictest15537-21ssh11:34:401
656599118,2cyclictest12534-21grep07:20:181
656599118,2cyclictest11143-21sed08:25:181
656199119,1cyclictest13-21ksoftirqd/011:46:180
656199118,2cyclictest6276-21ssh11:09:470
656199118,2cyclictest359-21ssh09:10:180
656199118,2cyclictest32046-21ssh10:07:220
656199118,2cyclictest2975-21grep09:50:160
656199118,2cyclictest26258-21ssh12:00:030
656199118,2cyclictest24312-21ssh09:43:470
656199118,2cyclictest23527-21ssh09:24:030
656199118,2cyclictest23464-21head11:00:180
656199118,2cyclictest23437-21head08:50:170
656199118,2cyclictest23249-21date10:40:280
656199118,2cyclictest22008-21grep07:40:180
656199118,2cyclictest17303-21sed07:30:140
656199118,2cyclictest12950-21ssh10:54:210
656199118,2cyclictest12581-21sh07:20:180
656199110,6cyclictest0-21swapper/010:45:110
657199108,1cyclictest8064-21grep07:10:203
657199108,1cyclictest30136-21sed07:55:283
657199108,1cyclictest197222sleep308:43:173
657199108,1cyclictest13774-21cat08:30:223
657199108,1cyclictest122972sleep308:27:583
657199107,2cyclictest5069-21taskset08:11:593
657199107,2cyclictest27497-21apt-get09:00:143
656799108,1cyclictest18413-21chrt07:32:122
656799107,2cyclictest20220-21grep07:35:222
656799107,2cyclictest14953-21hddtemp_smartct07:25:182
656799102,2cyclictest24780-21grep07:45:182
656799100,9cyclictest2460-21unixbench_multi08:05:272
656599108,1cyclictest29601-21chrt09:04:231
656599108,1cyclictest25511-21expr08:55:181
656599107,2cyclictest3830-21cut08:10:161
656599107,2cyclictest15066-21interrupts07:25:191
656599106,2cyclictest0-21swapper/108:45:121
656199108,1cyclictest8627-21tune2fs08:20:160
656199108,1cyclictest26383-21expr07:50:150
656199108,1cyclictest19694-21chrt08:42:540
656199108,1cyclictest15547-21expr08:35:150
656199107,2cyclictest9746-21expr07:15:150
656199107,2cyclictest4772-21wc08:10:250
656199107,2cyclictest29493-21sh10:25:220
656199107,2cyclictest27274-21date09:00:000
656199107,2cyclictest24748-21cat07:45:180
65679996,2cyclictest16338-21grep08:35:222
65659997,1cyclictest5839-21expr08:15:151
65659997,1cyclictest31744-21seq09:06:261
65619997,1cyclictest1797-21ls08:05:200
65619990,9cyclictest5958-21cat08:15:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional