You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-10 - 10:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Mar 10, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3767992523,1cyclictest5643-21ssh21:48:482
376599250,24cyclictest29467-21diskmemload23:07:201
376599250,24cyclictest0-21swapper/123:10:451
376599250,24cyclictest0-21swapper/122:49:501
376599250,24cyclictest0-21swapper/122:02:201
376599250,1cyclictest10112-21rm22:55:071
376499250,24cyclictest29467-21diskmemload23:07:250
376499250,24cyclictest29467-21diskmemload21:12:210
376499250,24cyclictest0-21swapper/021:45:160
377299240,24cyclictest0-21swapper/322:35:173
377299240,23cyclictest29467-21diskmemload21:48:463
377299240,23cyclictest28958-21ssh23:43:253
377299240,23cyclictest23760-21apt-get22:06:363
377299240,23cyclictest14637-21apt-get00:01:363
377299240,1cyclictest28065-21ssh23:11:413
3767992423,1cyclictest2954-21ssh23:18:392
376799242,21cyclictest16041-21ssh23:01:082
376799240,24cyclictest0-21swapper/221:31:362
376799240,23cyclictest29467-21diskmemload23:21:382
376799240,23cyclictest29467-21diskmemload22:20:142
3765992422,1cyclictest0-21swapper/121:40:451
376599240,24cyclictest0-21swapper/100:16:581
376599240,23cyclictest29467-21diskmemload23:45:451
376599240,23cyclictest29467-21diskmemload22:20:341
376599240,23cyclictest29467-21diskmemload22:14:011
376599240,23cyclictest29467-21diskmemload00:02:511
376599240,23cyclictest0-21swapper/122:52:081
376599240,23cyclictest0-21swapper/121:51:001
376599240,23cyclictest0-21swapper/121:28:211
3764992422,1cyclictest0-21swapper/023:16:360
376499240,24cyclictest0-21swapper/023:31:210
376499240,24cyclictest0-21swapper/022:17:450
376499240,24cyclictest0-21swapper/000:13:250
376499240,24cyclictest0-21swapper/000:03:370
376499240,23cyclictest0-21swapper/023:22:550
376499240,1cyclictest29467-21diskmemload21:35:440
3772992323,0cyclictest27127-21ssh22:40:273
3772992323,0cyclictest22856-21ssh21:34:253
3772992323,0cyclictest0-21swapper/322:45:233
3772992323,0cyclictest0-21swapper/321:22:223
377299232,21cyclictest20642-21ssh23:05:093
3772992321,1cyclictest20139-21ssh23:35:293
377299230,23cyclictest0-21swapper/300:08:593
377299230,22cyclictest9373-21sh23:25:033
377299230,22cyclictest8118-21ssh21:51:373
377299230,22cyclictest15243-21munin-run23:00:003
377299230,22cyclictest0-21swapper/300:16:363
377299230,0cyclictest0-21swapper/322:16:143
377299230,0cyclictest0-21swapper/321:35:483
377299230,0cyclictest0-21swapper/321:29:103
3767992323,0cyclictest29467-21diskmemload21:50:302
3767992323,0cyclictest0-21swapper/223:59:272
3767992323,0cyclictest0-21swapper/221:41:122
3767992321,1cyclictest28984-21apt-get22:11:362
3767992321,1cyclictest11603-21ssh22:25:352
3767992321,1cyclictest0-21swapper/223:05:572
3767992321,1cyclictest0-21swapper/200:31:062
376799230,2cyclictest27073-21idleruntime-cro21:05:012
376799230,23cyclictest0-21swapper/222:51:182
376799230,23cyclictest0-21swapper/222:06:422
376799230,22cyclictest31312-21grep23:46:372
376799230,22cyclictest29467-21diskmemload00:08:502
376799230,22cyclictest25431-21ssh23:40:382
376799230,22cyclictest19139-21ssh23:34:222
376799230,22cyclictest14474-21cstates21:26:382
376799230,1cyclictest24597-21ssh23:39:492
376799230,0cyclictest29467-21diskmemload00:35:272
376799230,0cyclictest0-21swapper/222:46:312
376799230,0cyclictest0-21swapper/221:58:122
376799230,0cyclictest0-21swapper/200:23:332
3765992323,0cyclictest0-21swapper/122:09:011
3765992323,0cyclictest0-21swapper/121:56:491
3765992322,1cyclictest29467-21diskmemload22:27:551
3765992322,1cyclictest0-21swapper/123:34:141
3765992322,1cyclictest0-21swapper/100:20:391
3765992321,2cyclictest23856-21sh23:38:331
3765992321,1cyclictest0-21swapper/123:04:191
3765992321,1cyclictest0-21swapper/121:17:391
3765992321,1cyclictest0-21swapper/121:13:311
376599230,2cyclictest17481-21cron20:45:001
376599230,23cyclictest0-21swapper/100:07:441
376599230,22cyclictest29467-21diskmemload23:21:331
376599230,22cyclictest26855-21head00:11:451
376599230,22cyclictest22404-21ssh22:36:191
376599230,22cyclictest0-21swapper/123:51:311
376599230,22cyclictest0-21swapper/123:41:371
376599230,22cyclictest0-21swapper/123:27:351
376599230,1cyclictest26906-21diskmemload22:40:081
376599230,1cyclictest0-21swapper/100:35:281
3764992323,0cyclictest0-21swapper/023:50:040
3764992323,0cyclictest0-21swapper/022:02:040
3764992323,0cyclictest0-21swapper/021:22:500
3764992321,1cyclictest31193-21diskmemload23:46:300
3764992321,1cyclictest27110-21ssh22:40:260
3764992321,1cyclictest0-21swapper/022:05:220
3764992321,1cyclictest0-21swapper/021:55:060
3764992320,2cyclictest14424-21apt23:30:010
376499230,23cyclictest29467-21diskmemload22:49:540
376499230,23cyclictest0-21swapper/023:03:550
376499230,23cyclictest0-21swapper/022:58:150
376499230,23cyclictest0-21swapper/022:13:060
376499230,23cyclictest0-21swapper/021:33:160
376499230,23cyclictest0-21swapper/000:25:230
376499230,22cyclictest0-21swapper/022:30:380
376499230,22cyclictest0-21swapper/021:16:480
376499230,22cyclictest0-21swapper/000:08:310
376499230,1cyclictest0-21swapper/000:15:430
376499230,0cyclictest0-21swapper/021:43:380
3772992222,0cyclictest29467-21diskmemload22:01:593
3772992222,0cyclictest0-21swapper/323:18:563
3772992222,0cyclictest0-21swapper/323:01:383
3772992222,0cyclictest0-21swapper/300:39:473
3772992221,1cyclictest29681-21apt-get21:41:373
3772992221,1cyclictest0-21swapper/322:22:173
3772992220,1cyclictest8246-21rm23:55:033
377299220,22cyclictest622-21gdbus00:25:013
377299220,21cyclictest4589-21sh23:20:393
377299220,21cyclictest3688-21ssh23:50:593
377299220,21cyclictest30380-21ssh23:45:073
377299220,21cyclictest18295-21ssh22:31:393
377299220,21cyclictest11927-21ssh22:26:043
377299220,21cyclictest0-21swapper/321:06:383
377299220,21cyclictest0-21swapper/320:06:393
377299220,21cyclictest0-21swapper/300:25:213
377299220,1cyclictest15118-21rm23:30:523
377299220,0cyclictest0-21swapper/322:52:123
377299220,0cyclictest0-21swapper/321:13:513
377299220,0cyclictest0-21swapper/300:10:083
3767992222,0cyclictest29467-21diskmemload22:40:352
3767992222,0cyclictest0-21swapper/221:11:582
3767992221,1cyclictest0-21swapper/200:10:112
376799220,22cyclictest0-21swapper/221:22:582
376799220,22cyclictest0-21swapper/200:25:032
376799220,22cyclictest0-21swapper/200:15:342
376799220,21cyclictest2610-21apt20:15:012
376799220,21cyclictest13652-21sh00:00:012
376799220,21cyclictest0-21swapper/223:51:592
376799220,1cyclictest3250-21ssh22:16:452
376799220,1cyclictest29467-21diskmemload22:01:372
376799220,1cyclictest26054-21sh22:39:172
376799220,0cyclictest29467-21diskmemload21:15:372
376799220,0cyclictest0-21swapper/223:25:022
376799220,0cyclictest0-21swapper/222:30:172
376799220,0cyclictest0-21swapper/221:36:332
3765992222,0cyclictest0-21swapper/121:20:131
3765992221,1cyclictest29467-21diskmemload21:45:341
3765992221,1cyclictest22661-21apt-get20:56:371
3765992221,0cyclictest0-21swapper/120:26:371
3765992220,1cyclictest0-21swapper/121:30:341
3765992220,1cyclictest0-21swapper/120:20:011
376599220,21cyclictest0-21swapper/119:26:351
376599220,21cyclictest0-21swapper/100:26:171
376599220,1cyclictest0-21swapper/123:55:501
376599220,1cyclictest0-21swapper/120:00:001
376599220,1cyclictest0-21swapper/100:33:231
376599220,0cyclictest0-21swapper/122:30:491
3764992222,0cyclictest0-21swapper/000:30:200
3764992220,1cyclictest0-21swapper/021:50:020
376499220,22cyclictest0-21swapper/000:23:370
376499220,21cyclictest29467-21diskmemload23:40:290
376499220,21cyclictest29467-21diskmemload22:36:130
376499220,21cyclictest0-21swapper/023:35:240
376499220,21cyclictest0-21swapper/022:25:470
376499220,21cyclictest0-21swapper/022:20:240
376499220,21cyclictest0-21swapper/021:01:380
376499220,21cyclictest0-21swapper/019:31:350
376499220,1cyclictest7445-21perf20:25:010
376499220,1cyclictest5073-21sh22:50:180
376499220,1cyclictest29467-21diskmemload23:55:460
376499220,1cyclictest25806-21apt-get19:56:350
376499220,1cyclictest23409-21apt-get19:51:350
376499220,1cyclictest0-21swapper/021:25:020
3772992121,0cyclictest29467-21diskmemload22:10:403
377299210,21cyclictest29467-21diskmemload21:15:283
377299210,21cyclictest0-21swapper/320:51:343
377299210,21cyclictest0-21swapper/319:46:363
377299210,1cyclictest15084-21lockfile-create20:40:003
377299210,0cyclictest2995-21apt-get20:16:373
377299210,0cyclictest0-21swapper/321:55:273
377299210,0cyclictest0-21swapper/321:01:373
3767992121,0cyclictest0-21swapper/222:55:262
3767992120,0cyclictest0-21swapper/219:11:352
3767992119,1cyclictest8539-21sendmail-msp19:20:012
3767992119,1cyclictest12269-21apt-get20:35:022
3767992119,1cyclictest0-21swapper/223:10:012
3765992121,0cyclictest0-21swapper/120:06:391
376599210,21cyclictest0-21swapper/120:31:361
376599210,20cyclictest13756-21apt-get19:31:351
376599210,20cyclictest0-21swapper/119:41:371
3764992120,0cyclictest0-21swapper/020:26:370
376499210,21cyclictest0-21swapper/019:43:370
376499210,0cyclictest4098-21apt-get19:11:350
3772992020,0cyclictest550-21apt-get20:11:363
3772992020,0cyclictest28222-21apt-get20:01:363
3772992020,0cyclictest10212-21apt-get20:31:363
377299200,0cyclictest0-21swapper/319:31:353
3767992020,0cyclictest0-21swapper/219:56:362
3767992018,1cyclictest24657-21sendmail21:00:012
376799200,0cyclictest0-21swapper/220:16:372
3765992020,0cyclictest0-21swapper/119:51:351
3765992020,0cyclictest0-21swapper/119:21:351
3765992019,1cyclictest29485-21perf21:10:011
376599200,20cyclictest0-21swapper/120:35:021
376599200,20cyclictest0-21swapper/119:11:341
376499200,0cyclictest0-21swapper/020:11:360
3772991918,1cyclictest0-21swapper/319:21:353
3772991917,1cyclictest0-21swapper/319:45:003
3767991919,0cyclictest0-21swapper/220:31:362
3764991919,0cyclictest0-21swapper/020:36:370
3764991919,0cyclictest0-21swapper/020:16:370
3764991919,0cyclictest0-21swapper/019:26:350
3767991818,0cyclictest0-21swapper/220:21:362
3767991815,2cyclictest9828-21ls20:30:002
3765991818,0cyclictest0-21swapper/120:11:361
3772991717,0cyclictest0-21swapper/319:11:353
376799174,12cyclictest26621-21munin-node19:23:022
3767991715,1cyclictest15473-21apt-get20:41:372
3764991717,0cyclictest0-21swapper/020:56:370
3764991717,0cyclictest0-21swapper/020:45:580
3772991616,0cyclictest22661-21apt-get20:56:373
3772991616,0cyclictest0-21swapper/320:41:373
3772991616,0cyclictest0-21swapper/319:56:353
3772991616,0cyclictest0-21swapper/319:51:353
3767991616,0cyclictest27471-21apt-get21:06:382
3767991616,0cyclictest0-21swapper/220:01:352
3767991615,1cyclictest18574-21apt-get19:41:372
3765991615,1cyclictest0-21swapper/119:36:361
376599160,15cyclictest20260-21apt-get20:51:341
3764991614,1cyclictest32633-21turbostat20:10:010
3764991614,1cyclictest201792sleep020:50:540
3764991614,1cyclictest0-21swapper/020:31:360
377299151,7cyclictest0-21swapper/320:21:363
3772991515,0cyclictest0-21swapper/319:36:363
377299150,14cyclictest17868-21apt-get20:46:363
3767991515,0cyclictest0-21swapper/220:46:362
3767991515,0cyclictest0-21swapper/219:51:352
376799150,1cyclictest0-21swapper/220:51:422
3765991515,0cyclictest0-21swapper/119:16:351
3764991513,1cyclictest8507-21cron19:20:010
3772991414,0cyclictest0-21swapper/320:28:183
377299140,14cyclictest0-21swapper/319:15:353
377299140,0cyclictest0-21swapper/319:26:353
376799140,14cyclictest0-21swapper/220:05:282
376799140,14cyclictest0-21swapper/219:35:062
376799140,14cyclictest0-21swapper/219:30:452
376799140,14cyclictest0-21swapper/219:27:532
376799140,0cyclictest0-21swapper/219:45:352
3765991414,0cyclictest0-21swapper/120:00:441
376599140,14cyclictest0-21swapper/120:47:021
3764991414,0cyclictest0-21swapper/020:00:390
3764991414,0cyclictest0-21swapper/019:35:050
376499140,0cyclictest0-21swapper/021:05:190
376499140,0cyclictest0-21swapper/020:40:560
376499140,0cyclictest0-21swapper/019:45:530
376499140,0cyclictest0-21swapper/019:20:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional