You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 22:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 09, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179792234161,59sleep00-21swapper/007:09:010
177852234168,54sleep10-21swapper/107:06:291
177652234164,58sleep30-21swapper/307:06:153
177242201164,25sleep20-21swapper/207:05:412
1669121620,5sleep31814899cyclictest08:15:253
2065921610,3sleep21814399cyclictest10:55:012
2065921610,3sleep21814399cyclictest10:55:012
2446421290,4sleep01813099cyclictest11:00:250
18143991081,105cyclictest0-21swapper/208:51:232
18143998783,3cyclictest0-21swapper/207:24:402
18143998783,3cyclictest0-21swapper/207:24:402
226982560,4sleep21814399cyclictest09:45:142
156802470,2sleep20-21swapper/209:30:012
1813099351,24cyclictest24158-21kworker/0:209:47:410
1813099342,10cyclictest514-21kworker/0:011:33:090
196262330,1sleep10-21swapper/112:05:111
1814899331,20cyclictest0-21swapper/311:30:493
1814899331,15cyclictest0-21swapper/311:56:363
1813099332,11cyclictest24158-21kworker/0:209:50:210
1813099321,11cyclictest6116-21kworker/0:108:05:120
1814899311,21cyclictest0-21swapper/308:20:093
1814899310,20cyclictest0-21swapper/309:26:133
1813099311,9cyclictest11158-21kworker/0:009:30:050
1814899301,13cyclictest0-21swapper/310:26:373
1814899301,12cyclictest0-21swapper/311:48:213
1814899301,12cyclictest0-21swapper/309:22:453
1814899300,16cyclictest0-21swapper/310:34:093
1814399300,13cyclictest0-21swapper/209:53:352
1813099302,15cyclictest20323-21munin-run07:15:010
1813099301,13cyclictest6853-21munin-run09:10:010
1813099301,10cyclictest11158-21kworker/0:009:25:090
18148992913,7cyclictest0-21swapper/310:48:053
1814899291,17cyclictest21803-21unixbench_singl09:40:253
1814899291,17cyclictest21803-21unixbench_singl09:40:253
1814899291,13cyclictest0-21swapper/307:35:413
1814899291,12cyclictest0-21swapper/312:07:293
1814399291,13cyclictest0-21swapper/208:42:512
1814899281,9cyclictest0-21swapper/308:42:293
1814899281,14cyclictest5841-21diskstats10:20:153
1814899281,12cyclictest0-21swapper/311:43:133
1814899280,12cyclictest0-21swapper/310:02:093
1814399281,3cyclictest0-21swapper/212:11:192
1814399281,12cyclictest0-21swapper/210:48:512
1814399281,11cyclictest724-21hddtemp_smartct12:35:152
1813099288,2cyclictest14502-21kworker/0:311:53:370
1813099282,9cyclictest19765-21kworker/0:110:05:160
1813099281,22cyclictest13788-21kworker/0:110:38:010
1814899271,13cyclictest0-21swapper/311:26:293
1814899271,13cyclictest0-21swapper/310:52:413
1814899271,13cyclictest0-21swapper/310:52:413
1814899271,13cyclictest0-21swapper/310:19:493
1814899271,12cyclictest6849-21diskmemload11:11:373
1814899271,12cyclictest0-21swapper/312:12:133
18143992714,8cyclictest24686-21vmstat11:00:262
1813099271,12cyclictest8928-21kworker/0:109:18:170
1813099271,11cyclictest13788-21kworker/0:110:58:170
1813099271,10cyclictest27064-21kworker/0:111:12:010
1813099270,18cyclictest6116-21kworker/0:107:58:000
1814899261,9cyclictest0-21swapper/308:33:053
1814899261,9cyclictest0-21swapper/308:33:053
18148992616,3cyclictest0-21swapper/311:00:333
1814899261,13cyclictest0-21swapper/308:08:373
1814899261,12cyclictest0-21swapper/311:54:123
1814899261,12cyclictest0-21swapper/311:24:133
1814899261,12cyclictest0-21swapper/311:08:333
1814899261,12cyclictest0-21swapper/308:58:013
1814399261,8cyclictest0-21swapper/207:16:102
1814399260,11cyclictest0-21swapper/212:24:432
1814399260,10cyclictest0-21swapper/211:37:232
1813099268,15cyclictest2462-21awk09:00:010
1813099266,5cyclictest6102-21kworker/0:211:37:490
1813099262,9cyclictest22582-21kworker/0:212:19:410
18130992611,2cyclictest25244-21kworker/0:107:25:530
60702250,1sleep10-21swapper/111:35:011
1814899251,16cyclictest0-21swapper/307:15:293
1814899251,12cyclictest0-21swapper/310:11:053
1814899250,16cyclictest0-21swapper/309:00:293
1814899250,12cyclictest0-21swapper/309:16:053
1814399251,11cyclictest0-21swapper/210:36:372
1814399251,10cyclictest0-21swapper/208:19:452
1814399250,11cyclictest0-21swapper/209:56:312
1814399250,11cyclictest0-21swapper/209:11:072
18134992512,10cyclictest13235-21kworker/u8:109:25:221
1813499251,16cyclictest330-21perl07:40:161
1813099251,9cyclictest1579-21kworker/0:107:42:490
1813099251,4cyclictest14900-21/usr/sbin/munin11:55:150
18130992512,3cyclictest3856-21kworker/0:207:52:320
1813099251,12cyclictest23822-21munin-run08:35:010
1813099251,12cyclictest23822-21munin-run08:35:010
1814899241,3cyclictest0-21swapper/309:33:053
1814899241,14cyclictest0-21swapper/308:46:333
1814899241,13cyclictest0-21swapper/309:46:013
1814899241,12cyclictest0-21swapper/309:58:173
1814899241,11cyclictest6849-21diskmemload09:14:213
1814899241,11cyclictest0-21swapper/312:36:293
1814899241,11cyclictest0-21swapper/312:03:213
1814899241,11cyclictest0-21swapper/310:37:533
1814399241,7cyclictest0-21swapper/209:40:472
1814399241,7cyclictest0-21swapper/209:40:472
1814399241,11cyclictest0-21swapper/211:48:552
1814399240,9cyclictest0-21swapper/209:15:152
1814399240,2cyclictest0-21swapper/211:11:552
1814399240,11cyclictest0-21swapper/209:32:592
1814399240,10cyclictest0-21swapper/210:22:512
1814399240,10cyclictest0-21swapper/209:24:552
18134992417,4cyclictest1176-21ntp_states12:35:201
1813099249,3cyclictest32186-21kworker/0:112:36:040
1813099248,2cyclictest7686-21kworker/0:310:29:530
1813099242,10cyclictest29781-21kworker/0:007:46:010
1813099241,4cyclictest0-21swapper/010:11:010
1813099241,15cyclictest7686-21kworker/0:310:34:570
18130992411,3cyclictest27905-21kworker/0:209:10:130
1813099241,10cyclictest12649-21kworker/0:008:28:250
18130992410,3cyclictest27478-21kworker/0:207:30:250
292422230,4sleep01813099cyclictest10:00:120
259702230,3sleep30-21swapper/307:25:143
1814899231,15cyclictest0-21swapper/312:26:533
1814899231,13cyclictest0-21swapper/310:07:413
1814899231,11cyclictest0-21swapper/312:34:133
1814899231,11cyclictest0-21swapper/310:42:533
1814899231,11cyclictest0-21swapper/307:48:293
1814399231,6cyclictest20379-21ntp_states12:05:202
1814399231,10cyclictest0-21swapper/207:29:032
1814399230,9cyclictest0-21swapper/210:04:392
1814399230,12cyclictest0-21swapper/210:56:112
1814399230,12cyclictest0-21swapper/207:30:192
1814399230,10cyclictest0-21swapper/211:06:512
1813499231,16cyclictest0-21swapper/107:48:481
1813099238,3cyclictest27622-21kworker/0:012:23:530
1813099236,2cyclictest17490-21kworker/0:309:35:290
1813099232,16cyclictest6116-21kworker/0:108:04:010
1813099231,11cyclictest27905-21kworker/0:208:47:130
1813099231,11cyclictest27905-21kworker/0:208:42:530
1813099231,10cyclictest12649-21kworker/0:008:16:090
1814899221,14cyclictest0-21swapper/311:35:053
1814899221,14cyclictest0-21swapper/308:01:093
1814399221,10cyclictest0-21swapper/209:37:352
1814399220,10cyclictest0-21swapper/210:40:192
1813499228,3cyclictest73252latency07:55:181
1813499222,15cyclictest0-21swapper/107:12:251
1813499221,9cyclictest29974-21memory10:00:191
1813499221,5cyclictest0-21swapper/109:51:471
1813499221,5cyclictest0-21swapper/109:32:431
1813499221,16cyclictest0-21swapper/107:34:261
1813499221,15cyclictest0-21swapper/111:59:471
1813499221,14cyclictest6849-21diskmemload11:23:481
1813499221,10cyclictest0-21swapper/107:15:571
1813499220,13cyclictest0-21swapper/110:38:451
1813099225,3cyclictest25394-21kworker/0:112:25:330
1813099223,8cyclictest19693-21grep10:50:210
1813099223,8cyclictest19693-21grep10:50:210
1813099222,6cyclictest27905-21kworker/0:209:01:250
1813099222,2cyclictest14502-21kworker/0:312:00:450
1813099222,10cyclictest28677-21kworker/0:010:18:250
1813099221,12cyclictest16702-21kworker/0:012:10:410
1813099221,10cyclictest514-21kworker/0:011:41:210
1813099221,10cyclictest12208-21kworker/0:211:49:370
1814899211,14cyclictest0-21swapper/310:56:493
1814899211,13cyclictest0-21swapper/312:22:493
1814899211,13cyclictest0-21swapper/308:53:013
1814899211,13cyclictest0-21swapper/307:32:133
1814899210,11cyclictest0-21swapper/311:17:533
1814399211,9cyclictest0-21swapper/208:07:592
18143992111,2cyclictest11879-21awk10:35:012
1814399210,9cyclictest0-21swapper/211:26:232
1813499218,4cyclictest31007-21ls12:30:171
1813499217,12cyclictest19925-21kworker/u8:009:41:301
1813499217,12cyclictest19925-21kworker/u8:009:41:301
1813499215,9cyclictest21-21rcuc/111:15:131
1813499212,3cyclictest0-21swapper/109:23:091
1813499212,17cyclictest0-21swapper/107:35:101
1813499211,18cyclictest0-21swapper/108:11:121
1813499211,18cyclictest0-21swapper/107:50:501
1813499211,17cyclictest26550-21cstates08:40:111
1813499211,11cyclictest8209-21latency_hist11:40:011
1813499211,11cyclictest26361-21ntp_states11:05:221
18134992110,5cyclictest0-21swapper/112:01:051
18134992110,3cyclictest23466-21cat12:14:561
1813499210,16cyclictest0-21swapper/111:10:401
1813499210,15cyclictest0-21swapper/111:45:171
1813099216,2cyclictest16702-21kworker/0:012:07:330
1813099214,5cyclictest0-21swapper/009:20:290
1813099214,4cyclictest31552-21kworker/0:107:15:130
1813099214,4cyclictest17146-21kworker/0:108:22:370
1813099212,9cyclictest25676-21kworker/0:008:37:290
1813099212,10cyclictest18759-21kworker/0:007:20:250
1813099212,10cyclictest18759-21kworker/0:007:20:250
1813099211,9cyclictest19765-21kworker/0:109:57:330
1813099211,10cyclictest27905-21kworker/0:208:52:450
1813099211,10cyclictest19765-21kworker/0:109:42:570
1813099211,10cyclictest19765-21kworker/0:109:42:570
1813099210,7cyclictest14794-21kworker/0:208:12:490
1814899201,13cyclictest0-21swapper/308:12:053
1814899201,12cyclictest0-21swapper/309:09:253
1814899201,12cyclictest0-21swapper/307:54:093
1814899201,12cyclictest0-21swapper/307:20:053
1814899201,12cyclictest0-21swapper/307:20:043
1814899201,11cyclictest24045-21/usr/sbin/munin08:35:173
1813499209,3cyclictest9918-21smart_sda08:00:251
1813499208,4cyclictest32616-21awk08:55:011
1813499202,17cyclictest0-21swapper/112:18:581
1813499201,4cyclictest87650irq/129-enp1s0-09:10:211
1813499201,3cyclictest0-21swapper/111:25:101
18134992012,6cyclictest14011-21kworker/u8:210:45:321
18134992012,3cyclictest0-21swapper/107:22:191
18134992012,3cyclictest0-21swapper/107:22:191
1813499201,17cyclictest28966-21kworker/u8:208:30:061
1813499201,17cyclictest28966-21kworker/u8:208:30:061
1813499201,17cyclictest0-21swapper/108:59:001
1813499201,14cyclictest0-21swapper/111:50:331
1813499201,11cyclictest9622-21df_inode09:15:151
1813499201,11cyclictest18985-21memory09:35:171
1813499201,10cyclictest29995-21unixbench_singl08:45:261
1813499201,10cyclictest0-21swapper/110:10:171
1813499200,2cyclictest87650irq/129-enp1s0-10:31:341
1813499200,18cyclictest0-21swapper/110:25:241
1813099204,2cyclictest514-21kworker/0:011:28:410
1813099203,4cyclictest5495-21/usr/sbin/munin10:20:280
1813099201,4cyclictest30803-21grep12:30:140
1813099201,11cyclictest514-21kworker/0:011:24:330
1813099201,11cyclictest0-21swapper/011:20:000
16202200,6sleep230-21rcuc/208:55:222
1814899192,8cyclictest0-21swapper/307:56:293
1814899191,12cyclictest0-21swapper/307:14:373
1814899191,11cyclictest0-21swapper/307:44:053
1814899191,10cyclictest19975-21runrttasks09:39:203
1814899191,10cyclictest19654-21/usr/sbin/munin08:25:283
1814899191,10cyclictest0-21swapper/312:17:513
1814399193,7cyclictest755-21rngd07:35:162
1814399191,8cyclictest12705-21/usr/sbin/munin11:50:122
1814399191,7cyclictest0-21swapper/210:12:182
18143991914,3cyclictest1493-21cat11:25:012
1814399191,15cyclictest30347-21gltestperf11:15:162
1814399191,12cyclictest0-21swapper/207:14:512
1813499198,3cyclictest4649-21latency_hist09:05:021
1813499196,5cyclictest0-21swapper/110:55:111
1813499192,16cyclictest0-21swapper/109:49:141
1813499191,9cyclictest0-21swapper/111:01:281
1813499191,7cyclictest0-21swapper/110:40:201
18134991915,3cyclictest0-21swapper/111:40:131
18134991915,3cyclictest0-21swapper/109:06:081
18134991915,3cyclictest0-21swapper/108:29:151
1813499191,4cyclictest5598-21cpuspeed_turbos10:20:141
18134991914,3cyclictest28834-40dnf10:00:011
18134991914,3cyclictest27873-21latency_hist12:25:021
1813499191,3cyclictest0-21swapper/108:05:161
1813499191,17cyclictest0-21swapper/108:16:321
1813499191,16cyclictest0-21swapper/110:17:111
1813499191,13cyclictest0-21swapper/110:54:481
1813499191,13cyclictest0-21swapper/110:54:481
1813499191,10cyclictest0-21swapper/112:25:231
1813499190,14cyclictest0-21swapper/107:26:231
1813099198,3cyclictest16485-21/usr/sbin/munin10:45:210
1813099196,4cyclictest29781-21kworker/0:007:35:370
1814899180,7cyclictest0-21swapper/309:50:173
1814399185,4cyclictest19420-21sh08:25:012
1814399185,3cyclictest9053-21timerandwakeup10:25:252
1814399185,3cyclictest0-21swapper/208:25:272
1814399182,4cyclictest18245-1kworker/u9:212:02:482
1814399181,6cyclictest30403-21cpuspeed_turbos12:30:122
1814399181,6cyclictest0-21swapper/212:28:352
1814399181,5cyclictest0-21swapper/210:15:302
1814399181,10cyclictest0-21swapper/212:16:392
18134991815,2cyclictest0-21swapper/108:20:081
1813499181,3cyclictest0-21swapper/110:05:251
1813499181,10cyclictest24281-21cstates08:35:121
1813099181,9cyclictest0-21swapper/011:10:020
1813099181,2cyclictest13788-21kworker/0:110:41:050
303322171,2sleep20-21swapper/208:49:132
1814399173,4cyclictest0-21swapper/208:13:012
1814399171,7cyclictest0-21swapper/209:09:222
1814399171,6cyclictest0-21swapper/211:55:062
1814399171,5cyclictest42832sleep209:00:312
1814399171,5cyclictest0-21swapper/210:05:082
18143991712,3cyclictest8959-21sed11:40:162
1814399171,12cyclictest0-21swapper/207:58:352
1814399171,10cyclictest0-21swapper/207:41:152
18143991710,4cyclictest24902-21sort08:35:182
1814399170,10cyclictest0-21swapper/211:30:032
1814399161,3cyclictest2868-21irqstats07:45:172
1814399161,2cyclictest0-21swapper/208:03:592
18143991611,3cyclictest23026-21processes08:30:222
18143991611,3cyclictest23026-21processes08:30:222
1814399160,10cyclictest0-21swapper/207:54:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional