You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-31 - 16:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Jul 31, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
248542233163,57sleep30-21swapper/307:08:543
247752215166,37sleep20-21swapper/207:07:522
227522214184,20sleep10-21swapper/107:05:011
248162202165,24sleep00-21swapper/007:08:230
2839821660,3sleep32504499cyclictest09:40:233
255152640,5sleep225518-21proc_pri08:20:232
25038995855,2cyclictest0-21swapper/208:26:262
65302450,2sleep20-21swapper/208:50:262
2502799438,32cyclictest25981-21awk12:05:020
2502799438,32cyclictest25981-21awk12:05:010
2502799397,14cyclictest6804-21awk12:35:010
25027993811,24cyclictest5582-21perf10:05:020
2502799361,11cyclictest29917-21sshd09:45:160
2502799357,13cyclictest21923-21basename08:15:000
2502799354,15cyclictest1794-21awk11:10:010
2504499341,24cyclictest0-21swapper/311:41:243
2502799344,3cyclictest0-21swapper/009:26:470
2502799344,14cyclictest808-21snmpd12:10:020
2504499330,26cyclictest0-21swapper/311:25:003
2502799336,12cyclictest17529-21idleruntime-cro08:05:000
2504499321,13cyclictest0-21swapper/310:17:433
2504499311,13cyclictest0-21swapper/311:19:553
2504499311,12cyclictest0-21swapper/312:09:033
2502799314,12cyclictest13752-21awk09:10:010
2504499301,18cyclictest0-21swapper/308:10:033
2504499301,11cyclictest0-21swapper/308:09:393
2504499300,2cyclictest0-21swapper/311:33:203
2504499300,11cyclictest0-21swapper/312:23:433
2504499291,12cyclictest0-21swapper/312:13:193
2504499291,12cyclictest0-21swapper/310:21:593
2504499291,11cyclictest0-21swapper/310:04:033
2503399291,20cyclictest0-21swapper/107:37:151
2503399290,21cyclictest0-21swapper/107:17:151
2503399290,21cyclictest0-21swapper/107:17:151
2502799293,13cyclictest28502-21munin-run08:30:010
2504499281,9cyclictest0-21swapper/310:45:153
2504499281,11cyclictest0-21swapper/310:35:433
2504499280,9cyclictest0-21swapper/310:44:313
2504499280,13cyclictest0-21swapper/308:28:373
2504499280,12cyclictest0-21swapper/310:31:553
2504499280,11cyclictest0-21swapper/309:50:593
2504499280,11cyclictest0-21swapper/308:33:353
2502799282,10cyclictest925-21kworker/0:112:27:580
2502799282,10cyclictest30374-21kworker/0:108:54:170
321212270,15sleep232-21ksoftirqd/211:05:022
2504499271,13cyclictest0-21swapper/309:35:443
2504499271,13cyclictest0-21swapper/309:35:443
2504499270,11cyclictest0-21swapper/312:35:513
2503399272,16cyclictest27160-21users10:50:261
2503399272,16cyclictest19555-21perl10:35:181
2502799272,13cyclictest3986-21munin-run11:15:010
2502799271,10cyclictest20677-21kworker/0:210:41:370
88592260,3sleep22503899cyclictest11:25:132
2504499265,3cyclictest0-21swapper/308:01:193
2504499261,10cyclictest0-21swapper/309:19:313
2504499260,9cyclictest0-21swapper/311:38:043
2504499260,9cyclictest0-21swapper/310:05:113
2504499260,13cyclictest0-21swapper/310:13:193
2504499260,12cyclictest0-21swapper/311:05:393
2504499260,12cyclictest0-21swapper/310:25:553
2504499260,11cyclictest0-21swapper/311:01:473
2504499260,10cyclictest0-21swapper/312:32:393
2503399263,14cyclictest16781-21/usr/sbin/munin10:30:121
2503399261,19cyclictest0-21swapper/110:46:031
2503399261,17cyclictest11255-21ntp_states10:15:221
2502799269,3cyclictest25599-21kworker/0:107:30:020
2502799262,12cyclictest29125-21munin-run09:45:020
2502799262,10cyclictest27409-21kworker/0:211:36:450
25027992611,3cyclictest25136-21kworker/0:110:50:130
2502799261,13cyclictest10573-21munin-run11:30:010
2502799261,12cyclictest29753-21perf11:00:010
2504499251,10cyclictest0-21swapper/308:15:393
2504499250,12cyclictest0-21swapper/309:02:153
2504499250,10cyclictest0-21swapper/311:11:033
2504499250,10cyclictest0-21swapper/309:27:443
2504499250,10cyclictest0-21swapper/309:21:123
2503399258,4cyclictest9994-21ntpq07:45:211
2503399251,16cyclictest2981-21entropy12:25:141
25033992511,12cyclictest13957-21kworker/u8:009:16:201
2502799259,3cyclictest15584-21kworker/0:109:15:170
2502799252,11cyclictest30297-21kworker/0:012:19:220
2502799252,10cyclictest3254-21kworker/0:110:06:460
2502799251,9cyclictest14642-21kworker/0:011:41:170
25027992511,3cyclictest14234-21kworker/0:110:34:010
2504499248,3cyclictest24563-21perl12:00:153
2504499248,3cyclictest24563-21perl12:00:153
2504499240,9cyclictest0-21swapper/311:29:433
2504499240,9cyclictest0-21swapper/307:33:393
2504499240,10cyclictest0-21swapper/309:45:053
2504499240,10cyclictest0-21swapper/309:11:273
25033992420,2cyclictest29128-21perf09:45:011
2503399241,17cyclictest0-21swapper/110:10:181
2503399241,15cyclictest13985-21strings11:35:191
2502799249,3cyclictest9760-21kworker/0:010:15:210
2502799249,3cyclictest1921-21kworker/0:007:41:340
2502799246,15cyclictest15947-21perf09:15:000
2502799242,10cyclictest10892-21kworker/0:008:19:530
25027992411,2cyclictest30297-21kworker/0:012:23:220
25027992411,2cyclictest24456-21kworker/0:209:35:330
25027992411,2cyclictest24456-21kworker/0:209:35:330
2502799241,10cyclictest6262-21kworker/0:307:47:260
2504499231,9cyclictest0-21swapper/307:43:153
2504499231,6cyclictest0-21swapper/307:45:433
2504499230,9cyclictest0-21swapper/312:26:353
2503399233,7cyclictest87550irq/128-enp1s0-10:00:201
2503399232,15cyclictest0-21swapper/111:20:341
25033992316,4cyclictest16166-21users10:25:271
2503399230,17cyclictest0-21swapper/112:10:241
2502799239,3cyclictest9280-21kworker/0:309:03:290
2502799238,3cyclictest4181-21kworker/0:207:35:140
2502799238,3cyclictest22893-21kworker/0:010:46:330
2502799237,4cyclictest14234-21kworker/0:110:29:290
2502799237,2cyclictest4216-21kworker/0:212:38:490
2502799235,2cyclictest14234-21kworker/0:110:24:570
2502799232,9cyclictest28998-21kworker/0:009:53:020
2502799232,9cyclictest25405-21kworker/0:108:24:250
2502799231,10cyclictest27409-21kworker/0:211:31:570
2502799231,10cyclictest21308-21kworker/0:311:58:210
25027992310,2cyclictest29641-21kworker/0:111:01:130
25027992310,2cyclictest14642-21kworker/0:011:45:370
2502799230,10cyclictest21308-21kworker/0:311:53:130
68642221,2sleep20-21swapper/210:05:232
2504499226,2cyclictest0-21swapper/308:55:233
2504499221,8cyclictest0-21swapper/309:31:483
2504499221,6cyclictest0-21swapper/310:59:273
2504499221,12cyclictest0-21swapper/307:50:073
2504499221,10cyclictest0-21swapper/307:56:313
2504499220,9cyclictest0-21swapper/309:05:193
2504499220,12cyclictest0-21swapper/311:58:563
2504499220,11cyclictest0-21swapper/309:55:473
2503399221,18cyclictest29547-21irqstats08:30:201
2503399221,15cyclictest0-21swapper/111:40:181
2503399221,14cyclictest0-21swapper/109:21:171
2502799228,2cyclictest28094-21kworker/0:212:10:140
2502799226,2cyclictest27409-21kworker/0:211:16:490
2502799221,17cyclictest0-21swapper/010:13:470
2502799221,10cyclictest27055-21kworker/0:307:21:210
2504499218,3cyclictest0-21swapper/310:54:473
2504499217,3cyclictest0-21swapper/307:24:313
2504499215,3cyclictest0-21swapper/308:48:153
25044992115,4cyclictest25822-21grep08:20:263
2504499210,9cyclictest0-21swapper/307:18:203
2504499210,9cyclictest0-21swapper/307:18:203
2503399217,12cyclictest30996-21kworker/u8:010:21:511
2503399215,3cyclictest165050irq/12-ttyS008:05:211
2503399214,9cyclictest15535-21/usr/sbin/munin08:00:121
2503399213,2cyclictest21-21rcuc/107:25:171
2503399212,14cyclictest0-21swapper/111:08:441
2503399211,9cyclictest0-21swapper/110:10:011
2503399211,18cyclictest0-21swapper/109:00:181
2503399211,17cyclictest4172-21sshd07:33:151
2503399211,14cyclictest0-21swapper/112:37:441
2503399211,13cyclictest0-21swapper/111:46:251
25033992110,3cyclictest21750-21perl10:40:161
2502799217,4cyclictest25599-21kworker/0:107:17:210
2502799217,4cyclictest25599-21kworker/0:107:17:210
2502799217,2cyclictest30374-21kworker/0:108:42:210
2502799217,2cyclictest14980-21kworker/0:107:59:100
2502799216,4cyclictest30374-21kworker/0:108:32:570
2502799216,2cyclictest4181-21kworker/0:207:35:020
2502799215,2cyclictest28998-21kworker/0:009:57:580
2502799212,11cyclictest18360-21/usr/sbin/munin09:20:270
2502799212,10cyclictest22291-21kworker/0:009:31:530
2504499206,2cyclictest0-21swapper/312:15:273
2504499201,12cyclictest0-21swapper/311:53:563
2503899205,6cyclictest13733-21diskmemload09:40:012
2503899205,6cyclictest13733-21diskmemload09:40:012
2503899201,7cyclictest0-21swapper/212:09:492
2503899201,6cyclictest0-21swapper/210:49:342
25038992015,3cyclictest21463-21grep08:10:262
2503899201,4cyclictest0-21swapper/211:31:192
2503399209,3cyclictest31939-21latency_hist11:05:011
2503399206,6cyclictest24724-21tune2fs08:20:161
2503399206,5cyclictest461-21grep09:50:251
2503399204,8cyclictest25590-21sshd07:10:161
2503399204,8cyclictest25590-21sshd07:10:151
2503399203,15cyclictest0-21swapper/111:26:491
2503399203,10cyclictest27065-21fschecks_time08:25:161
2503399202,3cyclictest0-21swapper/110:56:331
2503399202,16cyclictest0-21swapper/109:26:491
2503399202,16cyclictest0-21swapper/108:10:501
2503399202,16cyclictest0-21swapper/107:50:211
2503399202,15cyclictest4790-21perl11:15:161
25033992016,3cyclictest0-21swapper/109:05:511
25033992016,3cyclictest0-21swapper/108:52:351
25033992012,3cyclictest0-21swapper/112:00:331
25033992012,3cyclictest0-21swapper/112:00:331
2503399201,17cyclictest23-21ksoftirqd/109:35:461
2503399201,17cyclictest23-21ksoftirqd/109:35:461
2503399201,17cyclictest0-21swapper/108:59:081
2503399201,14cyclictest0-21swapper/111:52:511
2503399201,14cyclictest0-21swapper/111:31:351
2503399201,11cyclictest6084-21smart_sda12:30:241
2503399201,11cyclictest22471-21df08:15:151
2503399201,11cyclictest15988-21cat09:15:011
2503399200,14cyclictest0-21swapper/112:24:191
2502799209,2cyclictest30374-21kworker/0:108:49:010
2502799207,2cyclictest10892-21kworker/0:008:35:130
2502799204,4cyclictest27409-21kworker/0:211:24:290
25027992014,3cyclictest27241-21latency_hist07:15:010
25027992014,3cyclictest27241-21latency_hist07:15:010
2504499198,3cyclictest0-21swapper/308:51:513
2504499195,3cyclictest0-21swapper/311:45:043
2503899191,7cyclictest0-21swapper/211:53:262
2503899191,5cyclictest0-21swapper/210:32:202
2503899191,12cyclictest0-21swapper/212:14:552
2503899190,7cyclictest0-21swapper/210:37:572
2503399195,2cyclictest89050irq/133-enp2s0-07:55:251
2503399192,2cyclictest23-21ksoftirqd/109:30:251
2503399192,10cyclictest755-21rngd09:45:261
25033991915,3cyclictest0-21swapper/112:06:191
25033991915,3cyclictest0-21swapper/111:11:151
25033991915,3cyclictest0-21swapper/108:49:371
25033991914,3cyclictest1429-21smartctl08:40:171
2503399191,16cyclictest0-21swapper/108:37:271
25033991911,3cyclictest0-21swapper/111:55:261
25033991910,3cyclictest0-21swapper/112:15:181
2502799198,3cyclictest17743-21/usr/sbin/munin08:05:210
25027991913,4cyclictest11273-21tr07:50:150
15992190,3sleep20-21swapper/211:07:562
64272181,2sleep20-21swapper/212:30:302
2504499187,2cyclictest0-21swapper/307:29:353
25044991812,3cyclictest674-21/usr/sbin/munin08:40:223
2503899182,13cyclictest30456-21ls09:45:222
2503899181,7cyclictest24366-21fschecks_count12:00:142
2503899181,7cyclictest24366-21fschecks_count12:00:142
2503899181,7cyclictest0-21swapper/209:23:172
2503899181,7cyclictest0-21swapper/209:14:452
2503899181,6cyclictest0-21swapper/211:12:002
2503899181,6cyclictest0-21swapper/209:55:202
25038991810,4cyclictest9503-21grep10:10:252
25033991815,2cyclictest0-21swapper/109:55:081
2503399181,16cyclictest0-21swapper/107:40:351
2503399180,13cyclictest0-21swapper/107:24:491
2502799183,3cyclictest30374-21kworker/0:108:56:210
209212181,2sleep00-21swapper/010:39:560
2504499177,3cyclictest808-21snmpd08:37:433
2504499171,6cyclictest27403-21diskstats07:15:013
2504499171,6cyclictest27403-21diskstats07:15:013
2504499171,12cyclictest0-21swapper/307:39:433
2503899171,9cyclictest0-21swapper/207:24:032
2503899171,8cyclictest28962-21kernelversion07:15:182
2503899171,8cyclictest28962-21kernelversion07:15:182
2503899171,7cyclictest8295-21/usr/sbin/munin12:35:262
2503899171,7cyclictest4931-21latency_hist08:50:022
2503899171,7cyclictest25520-21expr07:10:142
2503899171,7cyclictest25520-21expr07:10:132
2503899171,7cyclictest0-21swapper/211:15:272
2503899171,7cyclictest0-21swapper/210:17:032
2503899171,7cyclictest0-21swapper/209:30:192
2503899171,6cyclictest0-21swapper/212:25:162
2503899171,6cyclictest0-21swapper/210:51:052
2503899171,6cyclictest0-21swapper/209:41:532
2503899171,5cyclictest23327-21grep11:55:242
25038991712,3cyclictest9108-21cstates07:45:122
25038991712,3cyclictest3588-21/usr/sbin/munin10:00:192
25038991712,3cyclictest2941-21gltestperf07:30:172
2503899171,15cyclictest0-21swapper/211:23:362
2503899171,15cyclictest0-21swapper/209:55:012
25038991711,4cyclictest16806-21cut09:15:162
2503899171,14cyclictest0-21swapper/209:25:082
2503899171,10cyclictest0-21swapper/212:15:532
25038991710,4cyclictest427-21grep12:20:102
2503899167,6cyclictest14332-21needreboot07:55:202
2503899166,3cyclictest87650irq/129-enp1s0-09:05:152
2503899166,3cyclictest30-21rcuc/208:35:122
2503899161,6cyclictest674-21/usr/sbin/munin08:40:122
2503899161,6cyclictest15636-21ntpq10:25:212
2503899161,6cyclictest15199-21/usr/sbin/munin11:40:252
2503899161,5cyclictest0-21swapper/210:55:042
2503899161,5cyclictest0-21swapper/207:50:152
2503899161,4cyclictest0-21swapper/210:20:332
2503899161,3cyclictest0-21swapper/208:31:362
25038991612,3cyclictest0-21swapper/208:05:172
25038991612,3cyclictest0-21swapper/207:40:082
25038991612,2cyclictest32512-21/usr/sbin/munin07:25:252
2503899161,14cyclictest0-21swapper/208:17:242
25038991611,3cyclictest8516-21proc_pri08:55:242
25038991611,3cyclictest15534-21/usr/sbin/munin08:00:152
25038991611,3cyclictest12991-21/usr/sbin/munin11:35:192
2503899160,15cyclictest0-21swapper/209:02:402
2503899159,3cyclictest21449-21expr10:40:132
25038991511,2cyclictest4549-21/usr/sbin/munin07:35:262
25038991510,3cyclictest18783-21cat11:45:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional