You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 17:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Jul 14, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2549129583,8sleep20-21swapper/207:07:152
2548929267,8sleep00-21swapper/007:07:130
2564728464,16sleep10-21swapper/107:09:151
2552528471,8sleep30-21swapper/307:07:403
2579399290,29cyclictest0-21swapper/209:11:302
2578799280,1cyclictest0-21swapper/012:16:470
2579399250,25cyclictest0-21swapper/211:06:092
2579399250,24cyclictest0-21swapper/209:23:492
2579199251,22cyclictest0-21swapper/109:10:131
2579399240,22cyclictest0-21swapper/211:30:202
2579399240,1cyclictest0-21swapper/209:15:422
2579399223,17cyclictest0-21swapper/210:02:352
2579399220,21cyclictest0-21swapper/212:33:482
2579399220,21cyclictest0-21swapper/212:22:352
2579399210,4cyclictest0-21swapper/209:56:062
2579399210,1cyclictest0-21swapper/212:29:192
2579399210,1cyclictest0-21swapper/210:10:382
2578799210,1cyclictest0-21swapper/009:57:310
2579399200,19cyclictest0-21swapper/211:42:382
2579399200,19cyclictest0-21swapper/209:48:092
25791992019,1cyclictest25-21ksoftirqd/111:02:321
2579199200,2cyclictest0-21swapper/111:18:561
2578799200,3cyclictest0-21swapper/010:10:370
2579399191,18cyclictest0-21swapper/210:31:542
2579399191,16cyclictest0-21swapper/211:46:202
2579399190,0cyclictest0-21swapper/211:36:582
2579199191,1cyclictest0-21swapper/111:58:501
2578799193,12cyclictest9923-21cron11:20:000
25787991918,1cyclictest0-21swapper/010:45:160
2578799190,19cyclictest0-21swapper/010:31:540
2579799183,2cyclictest0-21swapper/309:45:113
25797991816,1cyclictest0-21swapper/311:07:213
2579399182,15cyclictest0-21swapper/210:39:242
2579399180,17cyclictest0-21swapper/210:49:112
2579199182,2cyclictest18389-21diskmemload10:10:461
25791991816,2cyclictest38150irq/131-snd_hda11:42:131
2578799180,4cyclictest0-21swapper/009:41:220
2579399170,1cyclictest0-21swapper/211:28:562
2579399170,17cyclictest0-21swapper/211:56:392
2579399170,17cyclictest0-21swapper/209:33:312
25787991717,0cyclictest0-21swapper/012:31:000
2578799170,4cyclictest0-21swapper/011:05:530
2578799170,2cyclictest0-21swapper/009:45:070
2578799170,14cyclictest0-21swapper/010:18:420
25797991615,1cyclictest0-21swapper/312:32:243
25797991614,1cyclictest0-21swapper/310:32:153
25797991613,3cyclictest0-21swapper/309:30:183
25797991612,3cyclictest14275-21turbostat.cron12:40:003
25793991615,1cyclictest0-21swapper/209:35:402
2579399160,15cyclictest0-21swapper/209:41:232
2579199160,2cyclictest0-21swapper/112:10:141
2579199160,1cyclictest0-21swapper/110:40:441
2579199160,16cyclictest0-21swapper/110:32:091
2579199160,0cyclictest0-21swapper/112:06:551
2579199160,0cyclictest0-21swapper/112:06:551
25787991614,1cyclictest0-21swapper/009:54:570
2578799161,3cyclictest0-21swapper/011:04:450
2578799161,13cyclictest0-21swapper/011:21:420
2578799160,1cyclictest0-21swapper/011:12:020
2578799160,15cyclictest0-21swapper/009:12:420
2578799160,0cyclictest0-21swapper/010:57:420
2579799155,1cyclictest225-21systemd-journal09:39:163
25797991515,0cyclictest0-21swapper/312:05:023
25797991515,0cyclictest0-21swapper/312:05:013
25797991514,1cyclictest0-21swapper/311:19:423
25797991514,1cyclictest0-21swapper/310:08:343
2579799150,2cyclictest0-21swapper/309:50:123
2579399155,4cyclictest32541-21apache212:12:202
25793991514,1cyclictest0-21swapper/210:18:422
2579399150,15cyclictest0-21swapper/212:07:362
2579399150,15cyclictest0-21swapper/212:07:352
2579199158,6cyclictest27709-21grep08:20:161
2579199158,4cyclictest225-21systemd-journal08:00:161
2579199157,4cyclictest27237-21apache_processe09:15:111
25791991514,1cyclictest0-21swapper/110:18:261
25791991512,3cyclictest0-21swapper/109:45:151
2579199150,1cyclictest0-21swapper/109:40:161
2578799154,4cyclictest0-21swapper/012:25:140
2578799151,12cyclictest0-21swapper/009:15:130
25797991412,1cyclictest4631-21ssh10:56:463
2579399140,14cyclictest0-21swapper/211:54:272
2579399140,13cyclictest0-21swapper/210:54:112
2579199149,4cyclictest4703-21wc11:35:251
25791991410,3cyclictest2982-21sed08:35:201
2579199140,1cyclictest0-21swapper/112:20:081
2579199140,1cyclictest0-21swapper/111:34:011
2579199140,10cyclictest0-21swapper/111:21:291
2578799144,5cyclictest4861-21cstates11:55:150
25787991413,1cyclictest0-21swapper/010:08:330
2578799140,7cyclictest0-21swapper/007:50:150
2578799140,2cyclictest0-21swapper/012:20:140
2578799140,2cyclictest0-21swapper/009:35:400
25797991310,2cyclictest5696-21tr10:00:143
25797991310,2cyclictest4383-21cat09:20:193
25797991310,2cyclictest4099-21sed11:35:193
2579399139,3cyclictest27945-21grep11:10:272
25793991311,0cyclictest0-21swapper/208:50:272
25793991310,2cyclictest29165-21head07:15:162
25793991310,2cyclictest17179-21ssh11:05:002
2579399130,13cyclictest0-21swapper/209:27:292
2579199139,3cyclictest29007-21cron09:55:011
2579199139,1cyclictest20284-21tr09:30:151
2579199138,4cyclictest5975-21cat10:00:141
25791991310,2cyclictest26747-21grep07:10:161
25791991310,2cyclictest16562-21cut10:45:131
25791991310,2cyclictest10606-21ls08:50:261
2579199130,12cyclictest31318-21fschecks_time12:30:171
2578799134,3cyclictest22807-21awk10:30:010
25787991310,2cyclictest29983-21sort08:25:180
25787991310,2cyclictest0-21swapper/010:53:450
2578799130,2cyclictest0-21swapper/009:32:420
2579799129,2cyclictest9660-21grep07:40:273
2579799129,2cyclictest9391-21grep08:50:143
2579799129,2cyclictest488-21tr08:30:223
2579799129,2cyclictest30952-21grep10:15:153
2579799129,2cyclictest29445-21head11:50:183
2579799129,2cyclictest27539-21grep11:30:143
2579799129,2cyclictest26325-21grep11:10:153
2579799129,2cyclictest24224-21grep12:25:283
2579799129,2cyclictest19166-21grep09:10:133
2579799129,2cyclictest16813-21grep10:25:263
2579799129,2cyclictest13151-21sed08:55:283
2579799129,2cyclictest12753-21cat11:40:203
2579799128,3cyclictest22146-21grep11:45:263
2579799128,3cyclictest11584-21sh09:25:133
2579799124,7cyclictest0-21swapper/309:18:403
2579799121,2cyclictest13137-21grep12:00:153
2579399129,2cyclictest6131-21sshd12:15:162
2579399129,2cyclictest31477-21cron08:30:002
2579399129,2cyclictest21366-21seq09:50:142
2579399129,2cyclictest17045-21head09:05:182
2579399129,2cyclictest15335-21rm11:22:412
2579399129,2cyclictest15065-21tr10:25:152
2579399124,7cyclictest0-21swapper/210:44:552
25793991211,1cyclictest0-21swapper/207:31:222
25793991210,1cyclictest74752sleep210:20:162
2579399120,11cyclictest1490-21cron08:35:012
2579199129,2cyclictest32170-21grep10:35:131
2579199129,2cyclictest26229-21sed08:15:271
2579199129,2cyclictest25509-21grep10:50:191
2579199129,2cyclictest17948-21tr11:05:131
2579199129,2cyclictest1501-21sh08:35:001
2579199129,2cyclictest14403-21grep07:50:281
2579199129,2cyclictest13731-21wc09:00:011
2579199129,2cyclictest10470-21grep07:45:161
2579199128,3cyclictest1784-21head10:55:191
2579199127,2cyclictest29551-21chrt09:35:191
2579199127,1cyclictest7555-21grep12:35:181
25791991210,1cyclictest5099-21wc10:00:001
2579199120,0cyclictest0-21swapper/111:12:081
2578799129,2cyclictest9051-21sort10:40:200
2578799129,2cyclictest6729-21bc08:45:120
2578799129,2cyclictest5323-21ssh10:00:040
2578799129,2cyclictest30428-21sed12:10:190
2578799129,2cyclictest27710-21tail08:20:150
2578799129,2cyclictest17225-21wc09:05:200
2578799129,2cyclictest16223-21ssh12:01:310
2578799129,2cyclictest11825-21cut09:25:150
2578799129,2cyclictest10227-21head08:50:200
2578799122,2cyclictest0-21swapper/011:40:130
25787991210,1cyclictest4446-21wc07:30:210
2578799120,9cyclictest0-21swapper/009:22:540
2578799120,11cyclictest0-21swapper/012:35:130
2578799120,10cyclictest0-21swapper/010:35:100
2579799119,1cyclictest1205-21rm12:12:003
2579799118,2cyclictest8062-21taskset11:56:393
2579799118,2cyclictest8019-21aten2_r7power_p10:40:143
2579799118,2cyclictest5436-21ssh12:15:103
2579799118,2cyclictest4416-21tr08:40:153
2579799118,2cyclictest4292-21grep07:30:203
2579799118,2cyclictest30048-21tune2fs08:25:183
2579799118,2cyclictest29946-21cat07:15:263
2579799118,2cyclictest27645-21rm10:13:063
2579799118,2cyclictest22597-21expr08:10:143
2579799118,2cyclictest20710-21cat08:05:193
2579799118,2cyclictest15555-21expr07:55:143
2579799118,2cyclictest11065-21grep07:45:183
2579799118,2cyclictest10910-21ssh11:00:233
2579799117,3cyclictest0-21swapper/310:36:293
2579799116,2cyclictest225-21systemd-journal08:40:003
2579799110,9cyclictest0-21swapper/311:20:113
2579399119,2cyclictest685-21sh10:55:042
2579399119,2cyclictest6257-21ssh12:35:022
2579399119,1cyclictest2871-21grep11:15:162
2579399119,1cyclictest15838-21grep07:55:172
2579399119,1cyclictest15170-21sh12:00:422
2579399119,1cyclictest14960-21grep07:55:002
2579399119,1cyclictest14837-21cut09:00:182
2579399119,1cyclictest13907-21cut10:05:132
2579399118,2cyclictest9653-21sed07:40:262
2579399118,2cyclictest32030-21grep07:20:232
2579399118,2cyclictest29127-21turbostat08:25:012
2579399118,2cyclictest2624-21taskset07:26:172
2579399117,3cyclictest24324-21cron08:15:012
2579399115,2cyclictest225-21systemd-journal09:00:012
2579399115,2cyclictest225-21systemd-journal07:50:002
2579399114,4cyclictest55650irq/127-enp2s0-08:48:422
2579199118,2cyclictest930-21chrt11:52:371
2579199118,2cyclictest6871-21tr10:20:141
2579199118,2cyclictest5845-21sed08:40:271
2579199118,2cyclictest29510-21grep07:15:191
2579199118,2cyclictest24026-21chrt08:11:141
2579199118,2cyclictest20083-21ssh11:45:091
2579199118,2cyclictest19449-21sed11:25:161
2579199118,2cyclictest16553-21ssh12:01:461
2579199118,2cyclictest1608-21taskset07:25:181
2579199118,2cyclictest15776-21cut10:25:181
2579199118,2cyclictest15181-21cut07:55:141
2579199118,2cyclictest14785-21basename10:05:171
2579199118,2cyclictest11970-21tr09:25:161
2579199117,3cyclictest8803-21taskset08:49:291
2579199117,3cyclictest7468-21sed12:15:281
2578799119,1cyclictest8560-21sort07:40:160
2578799119,1cyclictest7424-21grep10:20:160
2578799119,1cyclictest3372-21cut11:35:150
2578799119,1cyclictest31844-21sh11:51:280
2578799119,1cyclictest31530-21grep07:20:180
2578799119,1cyclictest30068-21grep07:15:260
2578799119,1cyclictest264962sleep008:17:170
2578799119,1cyclictest26293-21tr07:10:130
2578799119,1cyclictest1808-21grep07:25:210
2578799118,2cyclictest825-21cat08:30:260
2578799118,2cyclictest31299-21sh11:32:040
2578799118,2cyclictest23498-21ssh12:05:390
2578799118,2cyclictest23498-21ssh12:05:390
2578799118,2cyclictest19924-21cut11:25:200
2578799118,2cyclictest11652-21grep07:45:240
2578799117,3cyclictest21027-21cut11:45:160
2578799113,2cyclictest0-21swapper/008:10:140
294242100,1sleep10-21swapper/108:25:161
2579799108,2cyclictest6962-21sh07:35:253
2579799108,1cyclictest6436-21rm10:20:063
2579799108,1cyclictest6263-21grep09:40:263
2579799108,1cyclictest29327-21cat08:25:013
2579799108,1cyclictest24998-21expr08:15:163
2579799108,1cyclictest2360-21grep07:25:283
2579799108,1cyclictest17886-21cat08:00:133
2579799108,1cyclictest16025-21ssh10:45:033
2579799108,1cyclictest15577-21grep09:00:263
2579799108,1cyclictest13685-21ssh12:20:033
2579799107,2cyclictest8847-21cron08:50:003
2579799107,2cyclictest31792-21cat07:20:213
2579799107,2cyclictest28702-21sh10:52:003
2579799107,2cyclictest16469-21grep09:05:163
2579799107,1cyclictest29269-21ssh09:55:023
2579399109,1cyclictest6017-21expr07:35:152
2579399109,1cyclictest246102sleep208:15:142
2579399108,2cyclictest18047-21sh08:00:142
2579399108,1cyclictest3336-21ls08:35:242
2579399107,2cyclictest4415-21grep08:40:142
2579399107,2cyclictest21963-21apt08:10:002
2579199108,1cyclictest6151-21ssh09:21:081
2579199108,1cyclictest24132-21grep12:25:261
2579199108,1cyclictest14948-21sort09:00:191
2579199107,2cyclictest3946-21sed07:30:161
2579199107,2cyclictest300-21taskset07:22:341
2578799109,1cyclictest6230-21grep07:35:180
2578799109,1cyclictest14494-21grep09:00:160
2578799108,1cyclictest36262sleep008:37:010
2578799108,1cyclictest18128-21cut08:00:160
2578799108,1cyclictest12976-21grep08:55:270
2578799105,4cyclictest21956-21sh08:10:000
257979998,1cyclictest26663-21tune2fs07:10:163
257979998,1cyclictest18594-21ssh11:25:023
257979998,1cyclictest145522sleep307:51:473
257939991,1cyclictest27176-21ntp_states07:10:202
257919997,1cyclictest20619-21head08:05:171
257919996,2cyclictest5825-21tr07:35:151
257919996,2cyclictest17037-21tail09:05:181
257919991,4cyclictest9052-21needreboot07:40:191
257879996,2cyclictest60892sleep008:42:530
257879996,2cyclictest15536-21expr07:55:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional