You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 21:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Aug 18, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14093992826,1cyclictest34-21ksoftirqd/312:21:443
14093992726,1cyclictest34-21ksoftirqd/310:27:583
14091992523,1cyclictest7438-21ssh09:36:162
1409199250,24cyclictest16219-1kworker/u9:209:10:122
1409199250,1cyclictest17810-21ssh12:36:232
1409199250,1cyclictest0-21swapper/209:45:542
14090992525,0cyclictest0-21swapper/109:19:251
14090992523,1cyclictest0-21swapper/111:12:501
1409099250,24cyclictest0-21swapper/111:39:531
14088992523,1cyclictest0-21swapper/012:22:210
1408899250,24cyclictest7818-21diskmemload10:03:090
1408899250,24cyclictest23263-21ssh12:13:360
1408899250,24cyclictest18924-21sh11:40:320
1408899250,1cyclictest0-21swapper/008:20:010
14093992424,0cyclictest34-21ksoftirqd/311:11:133
14093992423,1cyclictest34-21ksoftirqd/309:15:173
1409399241,22cyclictest34-21ksoftirqd/309:35:253
1409399240,24cyclictest34-21ksoftirqd/312:12:293
1409399240,24cyclictest0-21swapper/311:03:333
1409399240,23cyclictest0-21swapper/309:45:173
1409399240,1cyclictest34-21ksoftirqd/309:23:053
14091992423,1cyclictest0-21swapper/211:28:252
1409199242,21cyclictest16510-21pluginstate11:40:012
1409199240,23cyclictest7276-21cron10:35:012
1409199240,23cyclictest0-21swapper/212:33:002
1409199240,23cyclictest0-21swapper/211:58:042
1409199240,0cyclictest0-21swapper/211:15:162
1409099240,2cyclictest10044-21rm09:10:171
1409099240,23cyclictest7818-21diskmemload11:05:021
1409099240,23cyclictest7818-21diskmemload10:13:411
1409099240,23cyclictest28565-21awk07:40:001
1409099240,23cyclictest0-21swapper/111:15:491
1409099240,23cyclictest0-21swapper/110:45:391
1409099240,0cyclictest0-21swapper/111:52:381
1408899244,19cyclictest62550irq/127-eno110:27:200
14088992423,1cyclictest12393-21ssh09:13:090
1408899240,2cyclictest313-21systemd-journal08:55:010
1408899240,24cyclictest7818-21diskmemload11:55:100
1408899240,23cyclictest7818-21diskmemload09:42:440
1408899240,23cyclictest16179-21ssh12:06:370
1408899240,23cyclictest0-21swapper/011:27:350
1408899240,23cyclictest0-21swapper/010:18:050
1408899240,23cyclictest0-21swapper/009:29:300
1408899240,0cyclictest0-21swapper/009:20:540
14093992323,0cyclictest0-21swapper/312:31:173
14093992323,0cyclictest0-21swapper/311:59:453
14093992323,0cyclictest0-21swapper/310:07:313
14093992323,0cyclictest0-21swapper/309:26:133
14093992323,0cyclictest0-21swapper/307:30:023
14093992322,1cyclictest34-21ksoftirqd/311:40:033
14093992322,1cyclictest34-21ksoftirqd/310:55:303
14093992321,1cyclictest19710-21memory10:45:093
1409399230,23cyclictest0-21swapper/311:30:443
1409399230,23cyclictest0-21swapper/310:20:193
1409399230,22cyclictest7818-21diskmemload11:17:183
1409399230,22cyclictest7818-21diskmemload10:00:133
1409399230,22cyclictest6038-21ssh12:25:453
1409399230,22cyclictest27889-21ssh11:20:223
1409399230,22cyclictest15511-21sh12:05:373
1409399230,22cyclictest14261-21ssh10:11:013
1409399230,22cyclictest0-21swapper/310:16:373
1409399230,22cyclictest0-21swapper/309:53:053
1409399230,22cyclictest0-21swapper/309:40:183
1409399230,1cyclictest31855-21ssh11:51:433
1409399230,0cyclictest7818-21diskmemload11:25:373
1409399230,0cyclictest7818-21diskmemload11:08:203
1409399230,0cyclictest0-21swapper/312:18:543
14091992323,0cyclictest7818-21diskmemload10:05:222
14091992323,0cyclictest0-21swapper/212:00:122
14091992322,1cyclictest7818-21diskmemload09:21:452
14091992321,1cyclictest0-21swapper/210:15:032
14091992321,1cyclictest0-21swapper/209:55:062
14091992320,2cyclictest21414-21munin-run07:25:012
14091992320,2cyclictest17387-21ssh09:17:002
1409199230,2cyclictest18815-21dump-pmu-power07:20:012
1409199230,23cyclictest0-21swapper/212:16:332
1409199230,22cyclictest7818-21diskmemload10:50:242
1409199230,22cyclictest3345-21ssh10:01:272
1409199230,22cyclictest32329-21ssh11:52:292
1409199230,22cyclictest32123-21ssh10:27:202
1409199230,22cyclictest22193-21apt-get11:45:022
14090992323,0cyclictest7818-21diskmemload09:21:551
14090992323,0cyclictest0-21swapper/112:28:161
14090992323,0cyclictest0-21swapper/110:55:021
14090992323,0cyclictest0-21swapper/109:41:041
14090992323,0cyclictest0-21swapper/109:28:151
14090992322,1cyclictest20008-21ssh10:16:101
14090992321,1cyclictest0-21swapper/112:38:051
14090992321,1cyclictest0-21swapper/112:33:041
14090992321,1cyclictest0-21swapper/112:20:001
14090992321,1cyclictest0-21swapper/112:06:141
14090992321,1cyclictest0-21swapper/110:00:501
14090992321,1cyclictest0-21swapper/108:25:001
14090992320,1cyclictest0-21swapper/112:03:001
1409099231,1cyclictest16677-21apt-get07:15:011
1409099230,23cyclictest0-21swapper/111:48:261
1409099230,23cyclictest0-21swapper/109:35:131
1409099230,22cyclictest8177-21sh10:05:181
1409099230,22cyclictest7592-21ssh10:35:041
1409099230,22cyclictest6918-21ssh11:02:371
1409099230,22cyclictest4291-21sh10:30:591
1409099230,22cyclictest28835-21sh10:53:461
1409099230,22cyclictest25909-21apt-get11:20:031
1409099230,22cyclictest25210-21ssh09:52:321
1409099230,22cyclictest19095-21apt-get07:20:021
1409099230,22cyclictest0-21swapper/109:05:011
1409099230,1cyclictest0-21swapper/109:31:511
1409099230,0cyclictest0-21swapper/111:58:051
1409099230,0cyclictest0-21swapper/111:40:121
14088992323,0cyclictest14579-21apt-get11:10:020
14088992323,0cyclictest0-21swapper/010:30:340
14088992323,0cyclictest0-21swapper/010:08:200
14088992321,1cyclictest0-21swapper/011:30:050
14088992321,1cyclictest0-21swapper/010:38:250
14088992321,1cyclictest0-21swapper/009:55:000
14088992321,1cyclictest0-21swapper/009:32:350
14088992320,2cyclictest18835-21idleruntime-cro07:20:010
14088992320,2cyclictest0-21swapper/010:47:210
1408899231,1cyclictest62550irq/127-eno111:16:360
1408899230,2cyclictest5254-21ssh11:00:170
1408899230,23cyclictest0-21swapper/010:20:010
1408899230,22cyclictest28878-21ssh11:50:030
1408899230,22cyclictest28527-21turbostat.cron07:40:000
1408899230,22cyclictest19787-21ssh09:47:480
1408899230,22cyclictest17983-21sh12:36:360
1408899230,22cyclictest15249-21diskmemload10:40:340
1408899230,22cyclictest13428-21sh11:08:490
1408899230,22cyclictest13042-21idleruntime-cro12:05:010
1408899230,22cyclictest0-21swapper/008:00:030
1408899230,0cyclictest0-21swapper/009:17:130
14093992221,1cyclictest34-21ksoftirqd/309:58:313
14093992221,1cyclictest34-21ksoftirqd/308:15:013
14093992220,1cyclictest0-21swapper/310:50:083
1409399220,22cyclictest7818-21diskmemload10:31:123
1409399220,22cyclictest0-21swapper/312:37:173
1409399220,22cyclictest0-21swapper/309:14:083
1409399220,21cyclictest22852-21apt-get11:45:023
1409399220,21cyclictest15821-21ssh10:41:163
1409399220,21cyclictest15799-21apt-get08:20:013
1409399220,21cyclictest0-21swapper/312:00:233
1409399220,21cyclictest0-21swapper/310:35:263
1409399220,21cyclictest0-21swapper/309:05:023
1409399220,21cyclictest0-21swapper/307:20:023
1409399220,1cyclictest15799-21apt-get08:20:013
1409399220,1cyclictest1569-21ssh09:30:593
14091992222,0cyclictest0-21swapper/210:56:302
14091992221,1cyclictest8626-21ssh11:32:262
14091992220,1cyclictest0-21swapper/212:22:132
14091992220,1cyclictest0-21swapper/211:20:072
1409199220,21cyclictest8994-21apt-get11:05:032
1409199220,21cyclictest8840-21apt-get08:05:032
1409199220,21cyclictest7307-21rm11:03:062
1409199220,21cyclictest31533-21chrt09:30:012
1409199220,21cyclictest2649-21perf09:00:012
1409199220,21cyclictest15822-21latency_hist08:20:012
1409199220,21cyclictest13194-21apt-get10:40:042
1409199220,21cyclictest0-21swapper/211:44:402
1409199220,21cyclictest0-21swapper/211:10:432
1409199220,21cyclictest0-21swapper/210:35:452
1409199220,21cyclictest0-21swapper/210:10:452
1409199220,21cyclictest0-21swapper/209:52:312
1409199220,1cyclictest12804-21rm09:40:532
1409199220,0cyclictest0-21swapper/212:06:352
1409199220,0cyclictest0-21swapper/210:20:022
14090992221,1cyclictest31343-21ssh10:26:241
14090992220,1cyclictest32178-21ssh12:20:261
14090992219,2cyclictest607-21in:imuxsock09:10:001
1409099220,22cyclictest0-21swapper/112:10:021
1409099220,1cyclictest7818-21diskmemload10:43:341
1409099220,1cyclictest5218-21apt-get11:30:031
1409099220,1cyclictest26987-21apt-get09:55:001
1409099220,1cyclictest25483-21ssh10:20:571
1409099220,1cyclictest19708-21sh09:47:421
1409099220,1cyclictest0-21swapper/111:26:311
14088992222,0cyclictest0-21swapper/012:31:290
14088992221,1cyclictest62550irq/127-eno112:17:240
14088992221,1cyclictest5157-21ssh09:35:030
14088992221,1cyclictest30947-21latency_hist07:45:000
14088992220,2cyclictest1476-21ssh10:57:580
14088992220,1cyclictest28975-21ssh11:21:560
1408899220,21cyclictest611-21irqbalance11:45:430
1408899220,21cyclictest26859-21sh10:51:050
1408899220,21cyclictest13417-21munin-run08:15:010
1408899220,21cyclictest0-21swapper/008:35:020
1408899220,21cyclictest0-21swapper/008:25:030
1408899220,21cyclictest0-21swapper/007:55:010
1408899220,1cyclictest0-21swapper/012:25:020
1408899220,1cyclictest0-21swapper/010:12:430
14093992120,1cyclictest331rcuc/307:40:023
14093992120,0cyclictest0-21swapper/309:00:023
1409399210,20cyclictest0-21swapper/307:55:023
1409399210,0cyclictest0-21swapper/308:05:033
1409399210,0cyclictest0-21swapper/308:00:033
14091992120,0cyclictest0-21swapper/207:35:032
14091992119,1cyclictest28058-21apt-get08:45:022
1409199210,21cyclictest0-21swapper/209:00:022
1409199210,20cyclictest3634-21apt-get07:55:012
14090992121,0cyclictest0-21swapper/107:50:021
1409099210,21cyclictest0-21swapper/108:30:041
14088992121,0cyclictest0-21swapper/011:35:460
1408899210,0cyclictest14058-21apt-get07:10:010
1408899210,0cyclictest0-21swapper/007:50:000
14093992019,0cyclictest0-21swapper/308:50:033
14093992018,1cyclictest30208-21grep08:50:013
14091992020,0cyclictest0-21swapper/208:40:002
14091992020,0cyclictest0-21swapper/208:30:042
14091992020,0cyclictest0-21swapper/208:10:022
14090992020,0cyclictest0-21swapper/108:25:041
14090992020,0cyclictest0-21swapper/107:40:391
14090992018,1cyclictest16017-21latency_hist08:20:011
14088992020,0cyclictest0-21swapper/009:00:020
14088992020,0cyclictest0-21swapper/007:20:010
14093991919,0cyclictest0-21swapper/308:30:043
14093991918,1cyclictest34-21ksoftirqd/307:15:023
14091991918,0cyclictest30451-21apt-get08:50:032
14090991919,0cyclictest0-21swapper/108:40:001
14090991919,0cyclictest0-21swapper/108:05:031
14090991918,0cyclictest0-21swapper/108:50:031
14088991919,0cyclictest0-21swapper/008:20:020
14088991917,1cyclictest8840-21apt-get08:05:030
14093991818,0cyclictest0-21swapper/308:25:043
1409399180,0cyclictest0-21swapper/307:35:023
14091991818,0cyclictest14058-21apt-get07:10:012
14091991817,1cyclictest5695-21apt-get09:05:022
14091991816,1cyclictest23889-21apt-get07:30:022
14090991818,0cyclictest0-21swapper/107:25:011
1409099181,16cyclictest13377-21apt-get08:15:011
14088991818,0cyclictest0-21swapper/007:25:010
14088991816,1cyclictest0-21swapper/007:55:000
14093991717,0cyclictest0-21swapper/307:50:023
14091991717,0cyclictest0-21swapper/208:00:032
14091991717,0cyclictest0-21swapper/207:40:022
1409099170,2cyclictest30378-21latency_hist08:50:011
1409099170,0cyclictest6051-21apt-get08:00:031
14088991716,0cyclictest0-21swapper/008:45:020
14093991616,0cyclictest0-21swapper/307:10:013
14093991614,1cyclictest23246-21apt-get08:35:023
1409399160,1cyclictest901-21diskstats08:55:033
14090991616,0cyclictest0-21swapper/108:55:021
14088991616,0cyclictest0-21swapper/007:30:020
1408899160,15cyclictest0-21swapper/009:10:000
1409399150,1cyclictest0-21swapper/307:28:173
14091991515,0cyclictest0-21swapper/207:50:022
1409099150,1cyclictest0-21swapper/107:10:081
14088991514,0cyclictest0-21swapper/008:40:040
14093991414,0cyclictest0-21swapper/307:45:013
1409399140,0cyclictest0-21swapper/308:40:563
14091991413,1cyclictest18452-21apt-get08:25:032
1409199140,14cyclictest0-21swapper/208:40:162
1409199140,14cyclictest0-21swapper/208:20:302
1409199140,14cyclictest0-21swapper/207:45:212
1409199140,0cyclictest0-21swapper/207:25:052
14090991414,0cyclictest0-21swapper/107:56:031
1409099140,14cyclictest0-21swapper/108:42:201
1409099140,14cyclictest0-21swapper/107:45:201
14088991414,0cyclictest0-21swapper/008:55:020
14088991414,0cyclictest0-21swapper/008:30:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional