You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-17 - 01:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Mar 16, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32299992621,1cyclictest1728-21grep11:22:112
3229299260,2cyclictest20736-21sh11:08:520
3230299250,24cyclictest18388-21ssh10:05:003
3230299250,24cyclictest16692-21ssh12:39:523
32299992523,1cyclictest31038-21ssh09:46:042
3229999250,24cyclictest25699-21diskmemload09:12:132
3229699250,24cyclictest0-21swapper/112:02:191
3229299250,24cyclictest23362-21sh10:09:370
32302992423,1cyclictest8356-21runrttasks10:26:293
32302992423,1cyclictest6268-21ssh11:27:093
32302992423,1cyclictest25699-21diskmemload09:40:153
3230299240,23cyclictest25699-21diskmemload11:20:163
3230299240,23cyclictest25699-21diskmemload10:33:103
3230299240,23cyclictest24551-21rm11:43:473
3230299240,23cyclictest0-21swapper/311:02:443
3230299240,1cyclictest3948-21ssh11:55:433
3230299240,1cyclictest25699-21diskmemload10:57:113
32299992424,0cyclictest0-21swapper/209:28:312
32299992422,1cyclictest26421-21ssh11:14:392
32299992422,1cyclictest26421-21ssh11:14:392
32299992422,1cyclictest25327-21ssh09:40:172
3229999240,23cyclictest0-21swapper/210:49:402
3229999240,23cyclictest0-21swapper/210:24:232
3229999240,23cyclictest0-21swapper/210:24:232
3229699240,24cyclictest25699-21diskmemload10:04:461
3229699240,24cyclictest0-21swapper/109:22:591
3229699240,23cyclictest25699-21diskmemload10:47:101
3229699240,23cyclictest0-21swapper/109:34:021
3229699240,0cyclictest25699-21diskmemload10:24:381
3229699240,0cyclictest25699-21diskmemload10:24:381
3229699240,0cyclictest0-21swapper/112:31:081
32292992424,0cyclictest0-21swapper/009:55:550
32292992423,1cyclictest25699-21diskmemload11:29:070
32292992422,1cyclictest2500-21ssh12:26:240
32292992422,1cyclictest0-21swapper/009:28:340
32292992421,2cyclictest1051-21ssh11:22:060
32292992421,2cyclictest0-21swapper/011:17:160
3229299240,23cyclictest8829-21sh10:27:070
3229299240,23cyclictest25699-21diskmemload11:13:470
3229299240,23cyclictest25699-21diskmemload11:13:470
3229299240,23cyclictest25699-21diskmemload09:37:590
3229299240,1cyclictest62550irq/127-eno110:39:580
3229299240,1cyclictest25699-21diskmemload12:06:050
32302992323,0cyclictest25699-21diskmemload11:50:073
32302992323,0cyclictest0-21swapper/311:48:353
32302992323,0cyclictest0-21swapper/311:31:423
32302992321,1cyclictest25699-21diskmemload11:17:513
3230299230,23cyclictest0-21swapper/312:18:593
3230299230,23cyclictest0-21swapper/309:27:523
3230299230,22cyclictest8045-21rm12:31:543
3230299230,22cyclictest21272-21/usr/sbin/munin09:37:103
3230299230,22cyclictest1277-21awk08:20:003
3230299230,22cyclictest11681-21ssh09:57:323
3230299230,0cyclictest0-21swapper/311:37:463
3230299230,0cyclictest0-21swapper/310:42:093
3230299230,0cyclictest0-21swapper/309:22:253
32299992323,0cyclictest0-21swapper/212:22:072
32299992323,0cyclictest0-21swapper/211:57:092
32299992323,0cyclictest0-21swapper/211:27:262
32299992323,0cyclictest0-21swapper/210:56:282
32299992321,1cyclictest27293-21sh10:12:392
32299992321,1cyclictest0-21swapper/210:50:382
32299992320,2cyclictest11719-21ssh11:00:342
3229999231,1cyclictest20103-21apt-get10:07:092
3229999230,23cyclictest25699-21diskmemload09:18:302
3229999230,23cyclictest0-21swapper/212:30:472
3229999230,23cyclictest0-21swapper/212:15:522
3229999230,22cyclictest25699-21diskmemload11:43:552
3229999230,22cyclictest16218-21ssh11:36:302
3229999230,22cyclictest1542-21ssh11:52:212
3229999230,22cyclictest12627-21sh12:35:552
3229999230,22cyclictest0-21swapper/210:35:122
3229999230,22cyclictest0-21swapper/209:58:322
3229999230,1cyclictest10883-21ssh11:31:122
3229999230,0cyclictest0-21swapper/211:09:492
3229999230,0cyclictest0-21swapper/210:03:082
32296992323,0cyclictest0-21swapper/112:21:111
32296992323,0cyclictest0-21swapper/111:20:111
32296992322,1cyclictest31989-21ssh09:16:271
32296992321,1cyclictest28440-21ssh09:42:271
32296992321,1cyclictest0-21swapper/111:35:001
32296992320,2cyclictest14556-21fschecks_time10:32:121
3229699230,2cyclictest22668-21sh10:08:341
3229699230,23cyclictest0-21swapper/112:38:401
3229699230,23cyclictest0-21swapper/111:13:061
3229699230,23cyclictest0-21swapper/111:13:061
3229699230,23cyclictest0-21swapper/109:54:181
3229699230,22cyclictest25699-21diskmemload11:55:221
3229699230,22cyclictest25699-21diskmemload11:50:311
3229699230,22cyclictest25699-21diskmemload10:36:291
3229699230,22cyclictest25699-21diskmemload10:13:571
3229699230,22cyclictest19449-21ssh12:12:051
3229699230,22cyclictest12404-21awk10:30:011
3229699230,22cyclictest11629-21sh09:57:271
3229699230,22cyclictest0-21swapper/109:11:461
3229699230,1cyclictest0-21swapper/109:39:471
3229699230,0cyclictest25699-21diskmemload10:18:171
3229699230,0cyclictest0-21swapper/112:16:481
32292992323,0cyclictest25699-21diskmemload11:46:050
32292992323,0cyclictest25699-21diskmemload10:15:010
32292992323,0cyclictest0-21swapper/010:53:350
32292992322,1cyclictest9972-21ssh12:02:100
32292992321,1cyclictest30950-21ssh11:50:170
32292992321,1cyclictest0-21swapper/010:00:240
32292992321,1cyclictest0-21swapper/008:20:000
3229299230,23cyclictest25699-21diskmemload11:56:550
3229299230,23cyclictest0-21swapper/012:12:500
3229299230,23cyclictest0-21swapper/010:30:580
3229299230,23cyclictest0-21swapper/010:15:550
3229299230,23cyclictest0-21swapper/009:47:510
3229299230,22cyclictest25699-21diskmemload10:57:440
3229299230,22cyclictest25699-21diskmemload10:22:400
3229299230,22cyclictest25699-21diskmemload10:22:400
3229299230,22cyclictest25699-21diskmemload09:21:370
3229299230,22cyclictest16079-21ssh11:04:460
3229299230,22cyclictest15350-21ssh11:35:120
3229299230,22cyclictest0-21swapper/012:30:090
3229299230,22cyclictest0-21swapper/009:41:350
3229299230,21cyclictest26186-21rm09:10:300
3229299230,0cyclictest0-21swapper/011:40:030
32302992222,0cyclictest25699-21diskmemload10:07:393
32302992222,0cyclictest0-21swapper/312:21:043
32302992222,0cyclictest0-21swapper/312:09:223
32302992222,0cyclictest0-21swapper/309:16:373
32302992221,1cyclictest32136-21ssh10:48:293
32302992221,1cyclictest0-21swapper/312:13:143
32302992221,1cyclictest0-21swapper/310:13:593
32302992220,1cyclictest2564-21rm10:20:343
32302992220,1cyclictest2564-21rm10:20:343
32302992220,1cyclictest0-21swapper/309:35:013
32302992219,2cyclictest29085-21taskset10:15:073
3230299220,22cyclictest0-21swapper/310:52:583
3230299220,21cyclictest23742-21apt-get09:07:083
3230299220,21cyclictest18459-21sh10:36:053
3230299220,21cyclictest0-21swapper/311:10:013
3230299220,21cyclictest0-21swapper/311:10:013
3230299220,1cyclictest0-21swapper/312:25:123
3230299220,1cyclictest0-21swapper/309:50:203
3230299220,1cyclictest0-21swapper/307:40:003
32299992222,0cyclictest0-21swapper/212:25:402
32299992221,1cyclictest18478-21ssh12:10:222
32299992220,1cyclictest606-21dbus-daemon09:00:012
32299992220,1cyclictest0-21swapper/210:40:272
32299992220,1cyclictest0-21swapper/209:50:522
32299992220,0cyclictest0-21swapper/209:20:112
32299992219,2cyclictest29498-21sh10:15:442
32299992219,0cyclictest0-21swapper/210:26:052
3229999220,21cyclictest14845-21apt-get07:42:082
3229999220,21cyclictest0-21swapper/212:00:062
3229999220,21cyclictest0-21swapper/209:35:042
3229999220,21cyclictest0-21swapper/209:31:032
3229999220,1cyclictest27646-21idleruntime11:47:112
3229999220,1cyclictest0-21swapper/210:32:532
3229999220,0cyclictest0-21swapper/211:18:142
32296992220,1cyclictest4656-21latency_hist07:20:011
32296992220,1cyclictest22-21ksoftirqd/108:15:001
32296992220,1cyclictest0-21swapper/111:25:511
32296992220,1cyclictest0-21swapper/111:01:041
3229699220,22cyclictest0-21swapper/112:25:571
3229699220,21cyclictest19267-21perf07:50:011
3229699220,21cyclictest16857-21ssh11:05:331
3229699220,21cyclictest1251-21apt-get08:20:021
3229699220,21cyclictest0-21swapper/112:05:171
3229699220,21cyclictest0-21swapper/111:43:081
3229699220,21cyclictest0-21swapper/111:36:361
3229699220,21cyclictest0-21swapper/111:15:021
3229699220,21cyclictest0-21swapper/110:50:101
3229699220,21cyclictest0-21swapper/109:02:081
3229699220,21cyclictest0-21swapper/108:07:081
3229699220,21cyclictest0-21swapper/107:37:091
3229699220,1cyclictest8946-21apt-get09:25:021
3229699220,1cyclictest31067-21ssh09:46:081
3229699220,1cyclictest0-21swapper/111:46:041
3229699220,1cyclictest0-21swapper/110:55:551
32292992220,1cyclictest4061-21turbostat08:25:000
32292992220,1cyclictest20911-21apt09:00:010
3229299220,21cyclictest3207-21ssh09:50:130
3229299220,21cyclictest28921-21ssh12:20:410
3229299220,21cyclictest23667-21sh12:15:340
3229299220,21cyclictest0-21swapper/010:40:330
3229299220,21cyclictest0-21swapper/009:30:220
3229299220,21cyclictest0-21swapper/008:55:010
3229299220,1cyclictest606-21dbus-daemon07:50:000
3229299220,1cyclictest29269-21ssh10:46:500
3229299220,1cyclictest12742-21ssh12:36:060
3229299220,0cyclictest0-21swapper/009:19:020
32302992121,0cyclictest0-21swapper/312:00:293
32302992120,1cyclictest26891-21apt-get08:07:083
32302992117,3cyclictest9931-21apt-get07:32:103
3230299210,21cyclictest30881-21ssh09:45:503
3230299210,20cyclictest32682-21apt-get07:12:083
32299992121,0cyclictest29286-21apt-get08:12:102
32299992120,0cyclictest0-21swapper/208:22:082
32299992119,1cyclictest4659-21perf07:20:012
32296992121,0cyclictest0-21swapper/107:27:091
3229699210,20cyclictest0-21swapper/107:22:071
32292992121,0cyclictest0-21swapper/008:07:080
32292992120,0cyclictest7535-21apt-get07:27:090
32292992118,2cyclictest606-21dbus-daemon08:35:000
3229299210,0cyclictest0-21swapper/007:52:090
3229299210,0cyclictest0-21swapper/007:17:090
32302992020,0cyclictest0-21swapper/308:57:093
32302992020,0cyclictest0-21swapper/308:12:093
32302992020,0cyclictest0-21swapper/307:27:093
3230299200,20cyclictest0-21swapper/308:20:023
3230299200,20cyclictest0-21swapper/307:57:083
32299992020,0cyclictest16536-21apt-get08:52:082
32299992020,0cyclictest12327-21apt-get07:37:092
32299992020,0cyclictest0-21swapper/209:07:072
32299992020,0cyclictest0-21swapper/208:42:092
32296992018,1cyclictest14141-21apt-get08:47:081
3229699200,20cyclictest0-21swapper/108:57:091
32292992020,0cyclictest29286-21apt-get08:12:090
32292992020,0cyclictest0-21swapper/007:32:100
3229299200,20cyclictest0-21swapper/008:42:090
3229299200,0cyclictest5082-21apt-get07:22:080
32302991919,0cyclictest0-21swapper/309:02:083
32302991919,0cyclictest0-21swapper/308:52:093
32302991919,0cyclictest0-21swapper/308:37:093
32302991919,0cyclictest0-21swapper/307:47:093
32299991919,0cyclictest0-21swapper/208:27:082
32299991919,0cyclictest0-21swapper/208:02:112
32299991918,1cyclictest0-21swapper/208:32:092
32299991917,1cyclictest1267-21sendmail-msp08:20:002
32296991919,0cyclictest0-21swapper/108:37:091
32296991919,0cyclictest0-21swapper/108:02:111
32296991919,0cyclictest0-21swapper/107:32:101
32292991919,0cyclictest0-21swapper/007:57:080
32292991919,0cyclictest0-21swapper/007:37:090
32292991918,1cyclictest9331-21apt-get08:37:090
32292991917,1cyclictest23310-21cron09:05:000
32302991818,0cyclictest24501-21apt-get08:02:103
32299991818,0cyclictest0-21swapper/207:52:092
32296991818,0cyclictest0-21swapper/107:42:081
32292991818,0cyclictest0-21swapper/007:42:080
32302991717,0cyclictest0-21swapper/307:17:093
32299991716,0cyclictest0-21swapper/207:25:012
32299991715,1cyclictest24031-21cron08:00:012
32296991717,0cyclictest0-21swapper/108:52:081
32296991717,0cyclictest0-21swapper/107:57:081
32302991615,1cyclictest19683-21apt-get07:52:093
32299991616,0cyclictest0-21swapper/207:47:092
32296991616,0cyclictest11750-21apt-get08:42:091
32296991616,0cyclictest0-21swapper/107:12:081
3229699160,15cyclictest0-21swapper/109:07:071
32292991616,0cyclictest4482-21apt-get08:27:080
32302991515,0cyclictest0-21swapper/308:32:093
32299991513,1cyclictest9856-21perl08:37:112
32296991515,0cyclictest1266-21perf08:20:001
3229699150,5cyclictest0-21swapper/108:27:081
32292991515,0cyclictest0-21swapper/008:02:100
32302991414,0cyclictest0-21swapper/307:20:173
3230299140,14cyclictest0-21swapper/308:40:143
3230299140,14cyclictest0-21swapper/308:26:043
3230299140,14cyclictest0-21swapper/307:41:053
3230299140,0cyclictest0-21swapper/308:45:273
32299991414,0cyclictest0-21swapper/208:06:262
3229999140,14cyclictest0-21swapper/208:45:352
3229999140,14cyclictest0-21swapper/207:31:402
3229999140,14cyclictest0-21swapper/207:27:372
3229999140,0cyclictest0-21swapper/209:00:262
3229999140,0cyclictest0-21swapper/207:10:482
32296991414,0cyclictest0-21swapper/107:50:101
3229699140,0cyclictest0-21swapper/108:30:501
32292991414,0cyclictest0-21swapper/009:05:040
32292991414,0cyclictest0-21swapper/007:11:540
3229299140,14cyclictest0-21swapper/008:45:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional