You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 08:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Aug 11, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2446129481,8sleep00-21swapper/019:05:250
2485329367,8sleep20-21swapper/219:05:352
2511428963,8sleep30-21swapper/319:08:583
2515228568,13sleep10-21swapper/119:09:281
183082300,0chrt18309-21diskmemload00:22:512
2528699270,26cyclictest0-21swapper/022:55:300
2529399250,4cyclictest0-21swapper/222:25:442
2529699230,2cyclictest0-21swapper/321:31:013
25293992317,5cyclictest32-21ksoftirqd/223:40:012
2529399230,2cyclictest0-21swapper/222:34:292
2529399230,23cyclictest0-21swapper/222:19:282
2529399230,17cyclictest0-21swapper/222:02:042
107122230,0sleep10-21swapper/122:41:081
2529399220,22cyclictest0-21swapper/222:38:432
2529399220,22cyclictest0-21swapper/222:12:312
2528699220,21cyclictest0-21swapper/022:22:430
25296992117,3cyclictest39-21ksoftirqd/322:55:003
2528699210,20cyclictest0-21swapper/000:03:500
2529399200,20cyclictest0-21swapper/223:46:362
25296991915,3cyclictest8163-21seq21:22:413
2529199190,3cyclictest0-21swapper/122:09:511
5222180,0sleep20-21swapper/221:37:262
25293991817,1cyclictest0-21swapper/222:45:172
2529399180,17cyclictest0-21swapper/200:34:082
2529199181,1cyclictest18342-21diskmemload23:50:021
2529199180,18cyclictest5075-21kworker/1:222:24:421
2528699180,3cyclictest0-21swapper/022:48:490
2528699180,17cyclictest0-21swapper/000:25:470
2529699174,12cyclictest19313-21taskset22:27:203
25296991713,3cyclictest16732-21ssh23:05:003
2529399170,17cyclictest0-21swapper/223:56:442
2529199171,14cyclictest3641-21/usr/sbin/munin23:55:241
2528699170,1cyclictest0-21swapper/023:00:370
2529699163,13cyclictest0-21swapper/321:40:293
25296991612,2cyclictest12315-21ssh21:25:163
2529399160,1cyclictest0-21swapper/220:40:142
2529199163,6cyclictest0-21swapper/100:08:001
2528699163,7cyclictest26739-21munin-run21:15:000
2529699151,14cyclictest0-21swapper/323:18:553
2529699151,14cyclictest0-21swapper/323:18:553
2529699151,12cyclictest0-21swapper/300:05:583
2529399152,10cyclictest2353-21kworker/u8:100:12:012
25293991511,3cyclictest14877-21wc21:00:182
2529399150,12cyclictest0-21swapper/200:36:342
25291991515,0cyclictest0-21swapper/123:49:031
25291991511,2cyclictest0-21swapper/121:11:541
2529199150,14cyclictest0-21swapper/123:15:411
2529199150,14cyclictest0-21swapper/123:15:411
2528699157,7cyclictest18360-21perf21:10:000
2528699154,5cyclictest16915-21idleruntime-cro20:00:000
2528699152,7cyclictest29110-21perf20:25:000
2529699143,3cyclictest225-21systemd-journal21:55:003
2529699142,2cyclictest141rcu_preempt00:31:363
25296991410,4cyclictest90052sleep322:20:513
2529699140,13cyclictest0-21swapper/322:47:383
2529399149,1cyclictest12868-21grep00:20:002
2529399145,1cyclictest0-21swapper/222:05:122
2529399142,1cyclictest141rcu_preempt21:32:112
25293991410,3cyclictest8954-21wc19:40:262
2529399140,14cyclictest0-21swapper/221:56:282
2529199142,5cyclictest2251-21sshd21:38:441
2529199141,2cyclictest0-21swapper/122:35:061
2529199140,14cyclictest0-21swapper/122:29:121
2529199140,14cyclictest0-21swapper/100:18:581
2528699147,3cyclictest7188-21readlink19:40:000
2528699147,3cyclictest7188-21readlink19:40:000
25286991410,3cyclictest3340-21grep20:35:270
2529699139,3cyclictest17015-21head21:05:183
2529699139,2cyclictest1862-21cut20:35:143
2529699138,3cyclictest29087-21idleruntime-cro20:25:003
2529699131,10cyclictest0-21swapper/321:00:133
25296991310,2cyclictest4135-21wc23:35:213
25296991310,2cyclictest1073-21sh22:55:153
2529699130,13cyclictest0-21swapper/300:38:163
2529399139,3cyclictest25785-21tr19:10:142
2529399136,6cyclictest19372-21cstates21:10:152
2529399134,3cyclictest225-21systemd-journal20:02:012
2529399134,3cyclictest225-21systemd-journal00:06:592
2529399131,6cyclictest12712-21df_abs00:00:182
2529399131,3cyclictest141rcu_preempt20:10:012
25293991310,2cyclictest31064-21grep19:20:182
25293991310,2cyclictest2401-21uniq23:15:192
25293991310,2cyclictest2401-21uniq23:15:192
2529399130,2cyclictest0-21swapper/221:47:182
2529399130,13cyclictest0-21swapper/221:53:222
2529399130,13cyclictest0-21swapper/221:19:472
2529399130,12cyclictest0-21swapper/223:07:172
2529199139,3cyclictest23556-21cat20:10:261
2529199139,3cyclictest1093-21cron23:15:001
2529199135,6cyclictest26613-21/usr/sbin/munin23:30:231
2529199132,10cyclictest38150irq/131-snd_hda19:15:151
25291991310,2cyclictest9331-21tr20:50:151
25291991310,2cyclictest7763-21wc20:45:211
25291991310,2cyclictest29691-21tr21:55:131
25291991310,2cyclictest25977-21head22:50:241
25291991310,2cyclictest13470-21ssh21:25:261
25291991310,1cyclictest6133-21wc22:20:001
2529199130,12cyclictest0-21swapper/122:46:041
2528699139,4cyclictest21078-21sh22:09:370
2528699139,3cyclictest24845-21tr22:50:170
2528699139,3cyclictest16029-21cat21:05:000
2528699132,1cyclictest0-21swapper/021:46:010
2528699130,13cyclictest0-21swapper/022:38:590
2528699130,13cyclictest0-21swapper/021:37:490
2528699130,12cyclictest20121-21grep23:45:190
276002120,1sleep30-21swapper/300:28:343
2529699129,2cyclictest521-21tr19:25:133
2529699129,2cyclictest32433-21head20:30:163
2529699129,2cyclictest3240-21wc23:55:003
2529699129,2cyclictest3227-21grep19:30:173
2529699129,2cyclictest28556-21uniq19:15:183
2529699129,2cyclictest24266-21sed22:30:163
2529699129,2cyclictest23229-21tail20:10:203
2529699129,2cyclictest12820-21uniq00:00:203
2529699129,2cyclictest11554-21grep20:55:133
2529699129,2cyclictest10796-21sed20:50:263
2529699128,3cyclictest3851-21cat21:40:003
2529699128,3cyclictest13233-21munin-plugin-st22:05:013
2529699128,3cyclictest11903-21sed23:20:283
2529699127,4cyclictest39-21ksoftirqd/323:55:153
2529699127,4cyclictest29696-21grep21:55:133
2529699122,8cyclictest449-21irqbalance22:39:423
2529699121,6cyclictest225-21systemd-journal00:20:003
2529699120,11cyclictest0-21swapper/323:12:383
2529399129,2cyclictest2426-21grep22:55:252
2529399129,2cyclictest22880-21head20:10:172
2529399129,2cyclictest17459-21tr21:05:232
2529399129,2cyclictest15978-21sed19:55:212
2529399129,2cyclictest11741-21tr20:55:132
2529399128,3cyclictest361-21grep20:30:182
2529399127,4cyclictest311rcuc/223:41:582
2529399124,4cyclictest225-21systemd-journal23:05:002
2529399120,7cyclictest0-21swapper/222:20:022
2529199129,2cyclictest9726-21ssh22:02:161
2529199129,2cyclictest6547-21grep20:45:001
2529199129,2cyclictest29127-21grep00:10:141
2529199129,2cyclictest2750-21tr19:30:151
2529199129,2cyclictest22295-21grep00:25:151
2529199128,3cyclictest11962-21sed23:20:281
2529199128,2cyclictest31984-21tr20:30:131
2529199127,4cyclictest2340-21grep22:55:261
2529199127,4cyclictest12259-21cat20:55:161
2529199123,6cyclictest0-21swapper/100:35:141
2529199120,11cyclictest8941-21cut23:00:131
2528699129,3cyclictest0-21swapper/023:35:030
2528699129,2cyclictest8404-21cat19:40:190
2528699129,2cyclictest8127-21chrt21:41:480
2528699129,2cyclictest6759-21ls22:00:220
2528699129,2cyclictest5638-21cut00:15:180
2528699129,2cyclictest31966-21tr20:30:130
2528699129,2cyclictest29379-21ssh21:55:100
2528699129,2cyclictest29234-21grep21:15:290
2528699129,2cyclictest25760-21grep19:10:140
2528699129,2cyclictest25591-21ssh22:11:360
2528699129,2cyclictest25234-21grep20:15:190
2528699129,2cyclictest24756-21ssh00:07:100
2528699129,2cyclictest14699-21tr00:20:250
2528699129,2cyclictest13854-21wc19:50:260
2528699129,2cyclictest10527-21sed23:20:160
2528699128,3cyclictest8108-21tr22:40:150
2529699119,1cyclictest5047-21head20:40:183
2529699118,2cyclictest8859-21date20:50:003
2529699118,2cyclictest866-21ssh22:16:033
2529699118,2cyclictest8211-21head19:40:183
2529699118,2cyclictest28509-21ssh21:15:223
2529699118,2cyclictest26681-21dpkg23:30:133
2529699118,2cyclictest24618-21sh23:09:383
2529699118,2cyclictest23395-21ssh22:10:203
2529699118,2cyclictest23042-21ssh21:12:093
2529699118,2cyclictest21440-21ssh23:45:293
2529699118,2cyclictest20663-21ssh23:25:423
2529699118,2cyclictest16360-21ssh21:46:443
2529699118,2cyclictest15837-21ssh22:05:303
2529699118,2cyclictest1347-21sh00:12:383
2529699118,2cyclictest13137-21sh00:20:103
2529699118,2cyclictest12327-21ssh23:40:203
2529699116,2cyclictest8872-21irqstats22:40:193
2529399119,1cyclictest5307-21tr19:35:122
2529399119,1cyclictest5307-21tr19:35:122
2529399118,2cyclictest9722-21sh23:20:022
2529399118,2cyclictest7880-21ssh21:22:282
2529399118,2cyclictest7857-21apt-get22:40:132
2529399118,2cyclictest6965-21tr20:45:152
2529399118,2cyclictest5975-21rm21:40:232
2529399118,2cyclictest560-21grep19:25:142
2529399118,2cyclictest4623-21chrt19:33:042
2529399118,2cyclictest29283-21rm23:50:202
2529399118,2cyclictest28218-21sh23:30:232
2529399118,2cyclictest21470-21ssh00:25:102
2529399118,2cyclictest18439-21ssh23:25:142
2529399118,2cyclictest14325-21taskset19:52:402
2529399118,2cyclictest11425-21ssh21:25:042
2529399118,2cyclictest11018-21cut19:45:212
2529399117,3cyclictest25236-21tail20:15:192
2529399116,3cyclictest9679-21sort20:50:172
2529199118,2cyclictest7721-21grep19:40:151
2529199118,2cyclictest7084-21ssh21:40:591
2529199118,2cyclictest4873-21taskset23:35:371
2529199118,2cyclictest3787-21expr21:20:171
2529199118,2cyclictest30354-21apt19:20:141
2529199118,2cyclictest27757-21sed20:20:171
2529199118,2cyclictest26911-21mailstats19:10:521
2529199118,2cyclictest26625-21cron20:20:001
2529199118,2cyclictest24930-21ssh22:30:211
2529199118,2cyclictest22954-21grep21:50:261
2529199118,2cyclictest22722-21ssh22:10:151
2529199118,2cyclictest21350-21ssh23:26:151
2529199118,2cyclictest21084-21ssh23:06:521
2529199118,2cyclictest20412-21cut20:05:171
2529199118,2cyclictest1970-21grep20:35:151
2529199118,2cyclictest18286-21cut20:00:201
2529199118,2cyclictest17645-21wc21:05:261
2529199118,2cyclictest16721-21chrt19:57:551
2529199118,2cyclictest14814-21ssh21:45:321
2529199118,2cyclictest12962-21sed23:40:251
2529199117,3cyclictest26993-21ssh21:15:021
2529199117,2cyclictest12258-21ssh00:00:171
25291991110,1cyclictest0-21swapper/100:30:141
2529199110,10cyclictest17980-21ssh00:22:361
2528699119,1cyclictest23302-21ssh22:30:070
2528699119,1cyclictest22988-21tail21:50:270
2528699119,1cyclictest20030-21ssh21:30:140
2528699119,1cyclictest12869-21grep21:25:200
2528699118,3cyclictest27567-21apache222:28:040
2528699118,2cyclictest8832-21basename20:50:000
2528699118,2cyclictest8658-21ssh00:35:550
2528699118,2cyclictest568-21tr19:25:140
2528699118,2cyclictest4070-21sensors19:30:260
2528699118,2cyclictest3644-21tr21:20:170
2528699118,2cyclictest30763-21cat19:20:160
2528699118,2cyclictest3046-21grep23:15:240
2528699118,2cyclictest3046-21grep23:15:240
2528699118,2cyclictest3037-21ssh22:17:430
2528699118,2cyclictest29976-21ssh23:50:330
2528699118,2cyclictest27444-21cat23:10:250
2528699118,2cyclictest2381-21ssh00:13:270
2528699118,2cyclictest18782-21systemctl23:05:230
2528699118,2cyclictest18739-21cat23:25:150
2528699118,2cyclictest1248-21ssh00:31:470
2528699118,2cyclictest1143-21ssh23:34:140
2528699118,2cyclictest11390-21tr23:40:150
2528699113,1cyclictest225-21systemd-journal20:10:010
2528699110,10cyclictest19073-21taskset20:02:240
2529699108,1cyclictest30767-21cat19:20:163
2529699108,1cyclictest29649-21expr20:25:133
2529699108,1cyclictest27598-21taskset19:14:053
2529699108,1cyclictest26514-21chrt20:18:373
2529699108,1cyclictest19700-21cp20:05:013
2529699108,1cyclictest18983-21taskset20:01:153
2529699108,1cyclictest12803-21expr19:50:153
2529699108,1cyclictest10071-21grep19:45:143
2529699107,2cyclictest6086-21cat19:35:173
2529699107,2cyclictest6086-21cat19:35:173
2529699107,2cyclictest14820-21tr19:55:133
2529399108,1cyclictest1801-21aten2_r7power_p20:35:132
2529399107,2cyclictest28632-21tail20:20:272
2529399107,2cyclictest28058-21grep19:15:162
2529399107,2cyclictest25777-21cut23:10:142
2529399107,2cyclictest24902-21expr22:50:172
2529399102,4cyclictest225-21systemd-journal20:25:452
2529199108,1cyclictest6510-21sed19:35:231
2529199108,1cyclictest6510-21sed19:35:221
2529199108,1cyclictest371-21aten2_r7power_v19:25:131
2529199108,1cyclictest19808-21ssh21:30:051
2529199108,1cyclictest0-21swapper/121:00:251
2529199107,2cyclictest13072-21tune2fs19:50:171
2529199101,2cyclictest225-21systemd-journal20:25:001
2528699108,2cyclictest6221-21rm23:55:490
2528699108,1cyclictest9279-21cat20:50:140
2528699108,1cyclictest29746-21expr20:25:120
2528699107,2cyclictest27933-21acpi19:15:130
2528699107,2cyclictest10107-21grep19:45:140
2528699100,9cyclictest5203-21cat20:40:190
252919997,1cyclictest9816-21apache_accesses19:45:111
252869997,1cyclictest22971-21cat20:10:180
252869997,1cyclictest13028-21cat20:55:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional