You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 23:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Sep 06, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2379299261,24cyclictest141rcuc/010:42:430
23798992523,1cyclictest20025-21ssh11:07:453
23798992523,1cyclictest18347-21diskmemload11:33:373
23798992523,1cyclictest0-21swapper/312:30:033
2379899250,24cyclictest0-21swapper/311:13:333
2379899250,1cyclictest27529-21ssh10:46:043
2379499250,24cyclictest18347-21diskmemload11:20:402
23793992523,1cyclictest0-21swapper/111:26:161
2379399250,24cyclictest18347-21diskmemload11:41:181
2379299250,24cyclictest662-21sh10:51:100
2379299250,24cyclictest0-21swapper/010:14:190
2379299250,24cyclictest0-21swapper/009:37:090
2379899240,24cyclictest18347-21diskmemload11:51:563
23794992420,3cyclictest0-21swapper/209:59:562
2379499240,24cyclictest0-21swapper/211:15:042
2379499240,23cyclictest10294-21ssh10:01:152
23793992423,1cyclictest22-21ksoftirqd/109:48:431
2379399240,1cyclictest6294-21dpkg07:40:011
2379399240,0cyclictest0-21swapper/112:31:151
23792992424,0cyclictest0-21swapper/010:28:520
23792992424,0cyclictest0-21swapper/010:06:090
23792992423,1cyclictest18347-21diskmemload10:18:260
23792992423,1cyclictest18347-21diskmemload09:18:330
23792992423,1cyclictest1345-21apt-get07:31:360
23792992423,1cyclictest0-21swapper/011:24:390
2379299242,1cyclictest6281-21sendmail-msp07:40:010
23792992420,3cyclictest0-21swapper/011:06:390
2379299240,24cyclictest0-21swapper/009:10:170
2379299240,23cyclictest27690-21ssh10:46:170
2379299240,23cyclictest18347-21diskmemload09:58:300
2379299240,23cyclictest0-21swapper/012:20:020
2379299240,23cyclictest0-21swapper/012:01:260
2379299240,23cyclictest0-21swapper/011:51:380
2379299240,23cyclictest0-21swapper/010:37:550
2379299240,0cyclictest0-21swapper/009:48:190
23798992323,0cyclictest18347-21diskmemload11:58:043
23798992323,0cyclictest18347-21diskmemload09:49:033
23798992323,0cyclictest0-21swapper/312:16:593
23798992323,0cyclictest0-21swapper/312:01:333
23798992323,0cyclictest0-21swapper/310:34:263
23798992323,0cyclictest0-21swapper/310:10:503
23798992322,1cyclictest0-21swapper/310:57:163
2379899230,23cyclictest18347-21diskmemload09:41:573
2379899230,23cyclictest0-21swapper/310:17:483
2379899230,23cyclictest0-21swapper/309:32:303
2379899230,22cyclictest4741-21cron10:25:003
2379899230,22cyclictest3670-21diskmemload09:25:193
2379899230,22cyclictest31185-21ssh09:50:243
2379899230,22cyclictest23540-21ssh11:41:373
2379899230,22cyclictest23266-21rm09:13:343
2379899230,22cyclictest22424-21ssh10:41:263
2379899230,22cyclictest0-21swapper/312:27:033
2379899230,22cyclictest0-21swapper/312:21:373
2379899230,22cyclictest0-21swapper/311:03:593
2379899230,22cyclictest0-21swapper/310:36:143
2379899230,22cyclictest0-21swapper/310:05:593
2379899230,1cyclictest18347-21diskmemload10:50:263
2379899230,1cyclictest0-21swapper/309:16:513
2379899230,0cyclictest0-21swapper/311:18:123
23794992323,0cyclictest0-21swapper/212:22:362
23794992323,0cyclictest0-21swapper/211:59:512
23794992323,0cyclictest0-21swapper/210:17:272
23794992320,2cyclictest9282-21ssh11:26:572
23794992320,2cyclictest313-21systemd-journal11:00:002
23794992320,2cyclictest23582-21ssh09:42:362
23794992320,2cyclictest16239-21sh11:05:002
2379499230,22cyclictest25884-21ssh11:13:162
2379499230,22cyclictest25127-21ssh10:42:502
2379499230,22cyclictest24366-21ssh12:11:152
2379499230,22cyclictest19453-21ssh09:11:032
2379499230,22cyclictest18757-21sh12:34:572
2379499230,22cyclictest18347-21diskmemload10:23:152
2379499230,22cyclictest13379-21grep11:31:382
2379499230,22cyclictest11977-21ssh11:01:362
2379499230,22cyclictest1128-21ssh12:19:112
2379499230,21cyclictest18347-21diskmemload10:50:182
2379499230,1cyclictest18347-21diskmemload09:19:512
2379499230,0cyclictest18347-21diskmemload12:27:442
2379499230,0cyclictest15087-21rm10:05:192
2379499230,0cyclictest0-21swapper/210:45:072
2379499230,0cyclictest0-21swapper/210:33:442
23793992323,0cyclictest0-21swapper/112:20:021
23793992323,0cyclictest0-21swapper/111:52:001
23793992323,0cyclictest0-21swapper/111:12:181
23793992323,0cyclictest0-21swapper/110:36:351
23793992323,0cyclictest0-21swapper/109:31:091
23793992323,0cyclictest0-21swapper/109:10:201
23793992321,1cyclictest9150-21apt-get08:51:361
23793992321,1cyclictest0-21swapper/112:10:151
23793992321,1cyclictest0-21swapper/111:02:021
23793992321,1cyclictest0-21swapper/110:11:111
2379399231,1cyclictest11644-21apt-get10:31:361
2379399230,2cyclictest31344-21sh11:18:101
2379399230,23cyclictest0-21swapper/111:55:141
2379399230,23cyclictest0-21swapper/110:55:171
2379399230,23cyclictest0-21swapper/110:51:101
2379399230,23cyclictest0-21swapper/110:49:381
2379399230,22cyclictest28669-21ssh11:45:591
2379399230,22cyclictest18368-21wc09:10:011
2379399230,22cyclictest18347-21diskmemload11:36:191
2379399230,22cyclictest15564-21sh10:06:001
2379399230,22cyclictest13918-21perf07:55:001
2379399230,22cyclictest0-21swapper/110:02:411
2379399230,22cyclictest0-21swapper/109:56:171
2379399230,1cyclictest0-21swapper/111:30:261
2379399230,1cyclictest0-21swapper/109:50:161
2379399230,0cyclictest0-21swapper/112:00:501
2379399230,0cyclictest0-21swapper/110:41:541
2379399230,0cyclictest0-21swapper/109:29:511
23792992323,0cyclictest0-21swapper/012:36:280
23792992323,0cyclictest0-21swapper/011:56:590
23792992323,0cyclictest0-21swapper/009:41:540
23792992322,1cyclictest62550irq/127-eno112:12:130
23792992321,1cyclictest30490-21/usr/sbin/munin12:16:380
23792992321,1cyclictest20462-21ssh11:37:260
23792992321,1cyclictest0-21swapper/011:46:510
2379299231,21cyclictest0-21swapper/011:16:500
2379299231,21cyclictest0-21swapper/011:00:050
2379299231,21cyclictest0-21swapper/008:56:360
2379299230,22cyclictest11166-21apt08:55:000
2379299230,22cyclictest0-21swapper/012:30:030
2379299230,22cyclictest0-21swapper/011:42:060
2379299230,22cyclictest0-21swapper/010:22:370
2379299230,22cyclictest0-21swapper/009:30:590
2379299230,21cyclictest0-21swapper/011:32:520
2379299230,1cyclictest0-21swapper/007:55:000
2379299230,0cyclictest0-21swapper/011:26:160
2379299230,0cyclictest0-21swapper/010:55:170
2379299230,0cyclictest0-21swapper/009:20:390
23798992222,0cyclictest18347-21diskmemload10:27:123
23798992222,0cyclictest0-21swapper/312:38:123
23798992222,0cyclictest0-21swapper/312:12:233
23798992220,1cyclictest6582-21sh11:25:493
23798992220,1cyclictest31121-21apt-get08:30:033
23798992220,1cyclictest13567-21awk09:00:013
2379899220,21cyclictest18347-21diskmemload12:05:533
2379899220,21cyclictest0-21swapper/310:01:203
2379899220,1cyclictest28097-21ssh11:45:163
2379899220,1cyclictest2690-21ssh09:24:233
23794992222,0cyclictest21897-21apt-get08:11:362
23794992222,0cyclictest18347-21diskmemload12:01:542
23794992220,2cyclictest5168-21ssh10:25:162
23794992220,1cyclictest9468-21ssh09:30:452
23794992220,1cyclictest313-21systemd-journal08:10:002
23794992220,1cyclictest31102-21cron08:30:002
23794992220,1cyclictest22784-21ssh11:40:302
23794992220,1cyclictest18347-21diskmemload10:38:032
23794992220,1cyclictest0-21swapper/209:25:072
2379499220,22cyclictest0-21swapper/210:13:292
2379499220,22cyclictest0-21swapper/209:38:212
2379499220,21cyclictest6703-21apt-get08:46:372
2379499220,21cyclictest605-21avahi-daemon08:20:012
2379499220,21cyclictest31551-21ssh09:51:052
2379499220,21cyclictest16353-21apt-get09:06:362
2379499220,21cyclictest0-21swapper/211:51:142
2379499220,1cyclictest22895-21ssh12:37:592
2379499220,1cyclictest2191-21ssh09:23:362
2379499220,1cyclictest18149-21ssh12:05:242
23793992222,0cyclictest211rcuc/112:29:161
23793992222,0cyclictest0-21swapper/112:36:281
23793992222,0cyclictest0-21swapper/109:20:391
23793992221,1cyclictest0-21swapper/110:20:471
2379399220,22cyclictest27283-1kworker/u9:009:35:421
2379399220,22cyclictest18682-21kworker/u8:209:40:401
2379399220,22cyclictest0-21swapper/111:21:081
2379399220,21cyclictest3383-21perf07:35:011
2379399220,21cyclictest28977-21apt-get07:21:371
2379399220,21cyclictest0-21swapper/111:06:041
2379399220,1cyclictest0-21swapper/109:15:071
2379399220,0cyclictest0-21swapper/112:16:281
23792992222,0cyclictest0-21swapper/012:25:260
23792992220,1cyclictest0-21swapper/010:31:390
23792992220,1cyclictest0-21swapper/010:00:270
2379299221,0cyclictest18364-21apt09:10:010
2379299220,8cyclictest13-21ksoftirqd/007:44:010
2379299220,21cyclictest0-21swapper/011:12:260
2379299220,21cyclictest0-21swapper/009:50:370
2379299220,21cyclictest0-21swapper/009:01:380
2379299220,21cyclictest0-21swapper/008:36:350
2379299220,21cyclictest0-21swapper/008:11:360
2379299220,1cyclictest0-21swapper/012:05:020
2379299220,1cyclictest0-21swapper/009:26:140
23798992121,0cyclictest18347-21diskmemload09:36:013
23798992121,0cyclictest0-21swapper/311:21:343
23798992120,1cyclictest1891-21apt-get08:36:353
23798992120,0cyclictest0-21swapper/308:11:363
23798992120,0cyclictest0-21swapper/307:26:363
2379899210,20cyclictest0-21swapper/307:11:353
2379499210,21cyclictest0-21swapper/211:36:332
2379499210,21cyclictest0-21swapper/209:45:082
2379499210,0cyclictest31905-21apt-get08:31:362
2379499210,0cyclictest18347-21diskmemload11:47:222
23793992121,0cyclictest0-21swapper/109:01:381
23793992121,0cyclictest0-21swapper/107:42:271
2379399210,21cyclictest0-21swapper/108:07:041
2379399210,20cyclictest0-21swapper/110:15:091
23792992121,0cyclictest0-21swapper/007:56:360
2379299210,21cyclictest0-21swapper/008:46:370
2379299210,21cyclictest0-21swapper/008:05:020
2379299210,20cyclictest0-21swapper/008:16:350
23798992020,0cyclictest0-21swapper/309:01:383
23798992020,0cyclictest0-21swapper/308:06:363
23798992020,0cyclictest0-21swapper/307:41:363
23798992019,1cyclictest16713-21apt-get08:01:363
23798992018,1cyclictest6276-21ls07:40:013
23794992020,0cyclictest6273-21apt-get07:40:022
23794992020,0cyclictest0-21swapper/207:11:352
23794992018,1cyclictest13921-21turbostat.cron07:55:002
2379499200,20cyclictest0-21swapper/208:01:362
23793992020,0cyclictest0-21swapper/108:46:371
23793992020,0cyclictest0-21swapper/108:36:351
23793992020,0cyclictest0-21swapper/108:16:351
23793992020,0cyclictest0-21swapper/108:11:361
2379399200,20cyclictest0-21swapper/108:21:361
23792992020,0cyclictest11535-21munin-run07:50:000
23792992020,0cyclictest0-21swapper/007:21:370
23792992019,0cyclictest0-21swapper/008:21:360
2379299200,0cyclictest0-21swapper/008:26:350
23798991919,0cyclictest0-21swapper/307:56:363
23794991919,0cyclictest0-21swapper/208:51:362
23793991919,0cyclictest14297-21apt-get07:56:361
23793991919,0cyclictest0-21swapper/108:26:351
23793991919,0cyclictest0-21swapper/107:46:361
23793991914,4cyclictest18934-21munin-plugin-st08:05:011
23792991919,0cyclictest0-21swapper/008:41:390
23798991818,0cyclictest9150-21apt-get08:51:363
23798991818,0cyclictest0-21swapper/308:21:363
23798991817,1cyclictest9508-21apt-get07:46:373
23794991818,0cyclictest0-21swapper/207:56:362
23794991818,0cyclictest0-21swapper/207:26:372
23794991818,0cyclictest0-21swapper/207:16:352
23793991818,0cyclictest0-21swapper/108:41:391
23792991817,1cyclictest0-21swapper/008:01:360
23793991716,0cyclictest24160-21apt-get07:11:351
23792991717,0cyclictest0-21swapper/007:16:350
23792991717,0cyclictest0-21swapper/007:11:350
23792991716,1cyclictest0-21swapper/007:26:420
23798991613,2cyclictest6320-21sh08:45:003
23794991616,0cyclictest0-21swapper/207:21:372
2379499160,15cyclictest622-21gdbus07:35:012
2379499160,0cyclictest3370-21apt-get07:35:022
23793991614,1cyclictest12589-21/usr/sbin/munin08:56:411
23793991614,1cyclictest0-21swapper/107:26:421
23798991515,0cyclictest0-21swapper/308:46:373
23798991515,0cyclictest0-21swapper/308:16:353
23798991515,0cyclictest0-21swapper/307:51:363
23794991515,0cyclictest13964-21apt-get09:01:382
23794991514,1cyclictest4311-21apt-get08:41:392
23794991513,1cyclictest28723-21cron08:25:012
2379499150,1cyclictest0-21swapper/208:58:042
2379499150,14cyclictest2795-21memory08:36:402
23793991515,0cyclictest0-21swapper/107:16:351
23798991414,0cyclictest0-21swapper/307:30:243
2379899140,14cyclictest0-21swapper/309:05:093
2379899140,14cyclictest0-21swapper/308:25:083
2379899140,0cyclictest0-21swapper/307:20:303
2379899140,0cyclictest0-21swapper/307:15:113
2379499140,0cyclictest0-21swapper/207:45:152
2379399140,14cyclictest0-21swapper/108:30:091
2379299140,0cyclictest0-21swapper/008:32:420
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional