You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 19:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Dec 30, 2025 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283952233162,58sleep10-21swapper/107:08:261
281972206132,24sleep30-21swapper/307:05:513
284332203165,25sleep20-21swapper/207:08:562
265582200161,27sleep00-21swapper/007:05:070
28619991251,2cyclictest0-21swapper/211:14:162
2861999112108,3cyclictest0-21swapper/210:09:542
286199910494,9cyclictest0-21swapper/211:40:202
170632610,2sleep32862499cyclictest10:20:223
28619995350,2cyclictest0-21swapper/210:01:022
197182510,4sleep339-21rcuc/310:25:283
2860899431,13cyclictest13494-21kworker/0:310:45:130
2860899392,13cyclictest13494-21kworker/0:310:14:410
2860899392,11cyclictest3493-21kworker/0:212:18:220
28619993631,3cyclictest21073-21kworker/2:008:34:532
2860899362,11cyclictest2761-21kworker/0:209:55:570
28619993527,3cyclictest11411-21kworker/2:111:33:352
28619993420,3cyclictest11411-21kworker/2:112:28:092
2860899341,10cyclictest22054-21kworker/0:211:52:200
2860899331,10cyclictest24789-21kworker/0:011:56:520
28619993213,2cyclictest11168-21kworker/2:107:23:102
2860899321,11cyclictest22146-21kworker/0:110:33:510
2861999318,3cyclictest0-21swapper/207:30:202
2860899312,9cyclictest11678-21kworker/0:211:30:500
2860899312,17cyclictest28551-21kworker/0:110:50:290
28624993018,3cyclictest30288-21seq09:37:293
28619993010,2cyclictest21073-21kworker/2:008:10:082
2860899302,13cyclictest24789-21kworker/0:012:07:340
2860899301,11cyclictest5194-21kworker/0:209:15:400
2860899300,23cyclictest32665-21kworker/0:009:53:330
2861999294,10cyclictest0-21swapper/212:18:032
2861999294,10cyclictest0-21swapper/210:28:122
28619992912,4cyclictest11411-21kworker/2:112:05:372
28619992911,4cyclictest21073-21kworker/2:011:19:052
2860899296,2cyclictest30346-21kworker/0:209:38:520
2860899296,11cyclictest15127-21awk09:05:010
2860899296,11cyclictest15127-21awk09:05:010
2860899292,15cyclictest14275-21kworker/0:107:55:170
2860899292,10cyclictest13494-21kworker/0:310:38:010
2860899291,8cyclictest13494-21kworker/0:310:15:330
2860899291,6cyclictest0-21swapper/011:45:140
2860899291,22cyclictest25758-21kworker/0:209:27:360
2860899291,16cyclictest9431-21kworker/0:011:20:500
2860899291,12cyclictest28095-21kworker/0:109:40:060
2860899291,11cyclictest16112-21kworker/0:011:36:280
2860899291,10cyclictest24789-21kworker/0:012:24:460
2860899291,10cyclictest11678-21kworker/0:211:27:220
2862499282,3cyclictest0-21swapper/310:58:423
28624992818,3cyclictest0-21swapper/312:30:423
28624992816,7cyclictest0-21swapper/312:24:183
2861999288,2cyclictest11168-21kworker/2:107:48:462
2861999287,7cyclictest21073-21kworker/2:010:58:262
2861999287,3cyclictest0-21swapper/208:23:482
2861999284,4cyclictest10314-21df11:20:132
28619992818,2cyclictest21073-21kworker/2:008:15:282
28619992812,3cyclictest21073-21kworker/2:010:10:242
2860899288,3cyclictest30869-21kworker/0:010:56:110
2860899282,16cyclictest18407-21kworker/0:111:44:140
2860899280,11cyclictest5788-21kworker/0:112:26:240
28624992722,3cyclictest0-21swapper/311:33:363
28624992714,4cyclictest0-21swapper/308:10:103
2861999276,9cyclictest11411-21kworker/2:112:22:372
2861999275,5cyclictest11411-21kworker/2:111:36:392
2861999273,9cyclictest0-21swapper/211:05:562
28619992713,2cyclictest11168-21kworker/2:107:28:482
2860899272,12cyclictest5194-21kworker/0:208:53:480
2860899271,9cyclictest16374-21kworker/0:110:24:070
2860899271,6cyclictest0-21swapper/010:00:390
2860899271,24cyclictest5586-21kworker/0:211:13:530
28608992712,2cyclictest21042-21kworker/0:108:09:310
2860899271,17cyclictest21693-21kworker/0:009:23:360
2860899271,14cyclictest7800-21kworker/0:007:34:170
2860899271,11cyclictest31851-21kworker/0:008:33:160
2860899271,11cyclictest11108-21kworker/0:110:07:310
2862499268,3cyclictest9749-21latency_hist11:20:013
2862499262,4cyclictest0-21swapper/308:28:323
28624992619,3cyclictest0-21swapper/311:29:373
28624992619,3cyclictest0-21swapper/309:59:123
28624992617,2cyclictest0-21swapper/309:20:453
28624992616,3cyclictest0-21swapper/309:44:593
2861999264,10cyclictest0-21swapper/212:00:432
28619992615,3cyclictest21073-21kworker/2:009:51:382
2861499265,13cyclictest87150irq/128-enp1s0-12:15:361
2860899261,8cyclictest13494-21kworker/0:310:42:110
2860899261,6cyclictest0-21swapper/009:11:160
2860899261,5cyclictest0-21swapper/012:38:330
2860899261,5cyclictest0-21swapper/012:38:330
2860899261,5cyclictest0-21swapper/009:33:250
2860899261,23cyclictest0-21swapper/008:00:140
2860899261,11cyclictest3105-21kworker/0:111:08:050
2860899260,10cyclictest19789-21kworker/0:010:28:030
2862499252,2cyclictest87050irq/127-enp1s0-11:23:523
28624992519,2cyclictest0-21swapper/311:45:143
28624992519,2cyclictest0-21swapper/309:50:163
2862499251,2cyclictest87050irq/127-enp1s0-08:35:163
2861999259,2cyclictest21073-21kworker/2:009:04:392
2861999259,2cyclictest21073-21kworker/2:009:04:392
2861999258,8cyclictest11168-21kworker/2:107:54:562
2861999258,4cyclictest21073-21kworker/2:008:37:452
2861999254,7cyclictest0-21swapper/209:15:392
28619992512,2cyclictest11411-21kworker/2:111:54:572
28619992512,2cyclictest11168-21kworker/2:107:56:322
2861499251,7cyclictest21536-21kworker/u8:009:18:271
2861499251,17cyclictest0-21swapper/111:25:211
2860899259,2cyclictest5194-21kworker/0:208:59:560
2860899254,2cyclictest27361-21kworker/0:208:24:510
2860899252,3cyclictest7800-21kworker/0:007:45:230
2860899252,11cyclictest10324-21/usr/sbin/munin07:40:120
2860899251,6cyclictest0-21swapper/011:04:180
2860899251,5cyclictest0-21swapper/012:30:430
2860899251,5cyclictest0-21swapper/011:17:360
28608992512,2cyclictest24789-21kworker/0:012:00:160
28608992511,2cyclictest5194-21kworker/0:209:05:160
279492250,3sleep126942-21/usr/sbin/munin10:45:211
2862499244,3cyclictest7917-21chrt12:26:513
28624992421,2cyclictest0-21swapper/311:40:243
28624992417,3cyclictest0-21swapper/311:56:103
28624992417,3cyclictest0-21swapper/310:38:263
28624992417,3cyclictest0-21swapper/310:14:043
28624992417,3cyclictest0-21swapper/310:00:393
28624992417,3cyclictest0-21swapper/309:33:243
28624992417,3cyclictest0-21swapper/309:11:153
2862499241,4cyclictest0-21swapper/312:18:313
2861999249,5cyclictest21073-21kworker/2:008:46:072
2861999248,8cyclictest11168-21kworker/2:107:11:212
2861999248,7cyclictest11411-21kworker/2:111:45:532
2861999247,9cyclictest21073-21kworker/2:008:28:132
2861999241,4cyclictest30504-21munin-run09:40:022
2861999241,4cyclictest0-21swapper/208:44:372
28619992412,3cyclictest21073-21kworker/2:009:09:432
28619992410,2cyclictest11168-21kworker/2:107:17:592
2861499240,12cyclictest0-21swapper/111:39:031
2860899241,11cyclictest466-21/usr/sbin/munin09:45:140
2860899241,11cyclictest14716-21/usr/sbin/munin07:50:130
2862499236,11cyclictest0-21swapper/312:35:143
2862499236,11cyclictest0-21swapper/312:35:143
28624992316,3cyclictest0-21swapper/310:52:113
28624992316,3cyclictest0-21swapper/310:33:003
2862499231,5cyclictest30845-21ntp_states12:05:233
2861999239,8cyclictest21073-21kworker/2:010:50:302
2861999237,4cyclictest11168-21kworker/2:107:38:562
2861999236,8cyclictest87250irq/129-enp1s0-12:33:292
2861999236,5cyclictest21073-21kworker/2:009:33:252
2861999234,16cyclictest21612-21smart_sda10:30:252
2861999232,7cyclictest19111-21/usr/sbin/munin08:00:212
2861999231,3cyclictest0-21swapper/209:21:502
28619992312,3cyclictest21073-21kworker/2:009:26:332
2861499232,3cyclictest10712-21munin-run08:55:001
2861499232,11cyclictest30817-21munin-node07:15:011
2861499231,6cyclictest0-21swapper/112:37:261
2861499231,6cyclictest0-21swapper/112:37:261
2861499231,6cyclictest0-21swapper/109:43:111
28614992310,3cyclictest0-21swapper/109:48:061
2860899232,10cyclictest31693-21kworker/0:207:15:120
2860899231,4cyclictest0-21swapper/012:10:130
2860899231,19cyclictest801-21snmpd08:38:150
2860899231,19cyclictest4348-21/usr/sbin/munin08:40:260
2860899231,10cyclictest801-21snmpd07:38:040
28608992310,3cyclictest21042-21kworker/0:108:10:250
151882231,2sleep20-21swapper/210:15:252
28624992217,2cyclictest0-21swapper/310:15:213
28624992215,3cyclictest0-21swapper/312:10:123
28624992215,3cyclictest0-21swapper/310:05:053
28624992212,6cyclictest0-21swapper/309:17:593
2862499221,1cyclictest0-21swapper/311:52:133
2862499220,4cyclictest0-21swapper/308:46:303
2861999229,7cyclictest0-21swapper/208:57:312
2861999227,3cyclictest0-21swapper/210:22:362
2861999221,6cyclictest577-21systemd-journal08:55:002
28619992212,3cyclictest21073-21kworker/2:010:35:442
28619992210,2cyclictest21073-21kworker/2:011:01:542
2861499224,2cyclictest87150irq/128-enp1s0-10:25:231
2861499222,10cyclictest11954-21munin-run11:25:001
2861499221,6cyclictest0-21swapper/111:42:431
2861499221,4cyclictest29167-21kworker/u8:212:04:351
28614992214,3cyclictest0-21swapper/107:47:311
28614992213,2cyclictest5877-21cat11:10:121
2861499221,16cyclictest0-21swapper/110:14:551
2861499221,15cyclictest17308-21diskmemload12:33:061
2861499221,15cyclictest0-21swapper/108:09:251
2861499221,15cyclictest0-21swapper/107:57:121
2861499221,11cyclictest11449-21pmu-power07:40:221
2861499220,5cyclictest0-21swapper/110:44:261
2860899221,11cyclictest3159-21kworker/0:107:27:050
2860899220,9cyclictest26261-21kworker/0:007:10:420
2860899220,11cyclictest0-21swapper/008:25:300
2862499215,2cyclictest0-21swapper/310:45:143
2862499214,3cyclictest0-21swapper/307:38:563
2862499212,2cyclictest87050irq/127-enp1s0-11:05:383
28624992116,2cyclictest0-21swapper/311:04:193
28624992115,2cyclictest0-21swapper/311:15:023
2862499211,2cyclictest87050irq/127-enp1s0-08:09:163
2862499210,3cyclictest0-21swapper/309:05:313
2861999215,3cyclictest0-21swapper/212:14:432
2861999214,3cyclictest21073-21kworker/2:010:42:562
2861999211,6cyclictest0-21swapper/209:55:542
2861999211,4cyclictest0-21swapper/212:35:402
2861999211,4cyclictest0-21swapper/212:35:392
28619992110,3cyclictest21073-21kworker/2:010:50:022
2861499213,6cyclictest2661850kworker/u8:210:50:441
2861499212,5cyclictest30421-1kworker/u9:209:36:581
2861499212,5cyclictest30421-1kworker/u9:209:26:101
2861499212,3cyclictest17308-21diskmemload10:21:021
2861499211,8cyclictest0-21swapper/108:19:561
28614992117,3cyclictest0-21swapper/110:37:351
28614992116,3cyclictest32082-21sh08:30:011
2861499211,3cyclictest0-21swapper/112:09:051
28614992113,3cyclictest0-21swapper/108:37:101
28614992113,3cyclictest0-21swapper/107:19:521
2861499211,15cyclictest0-21swapper/112:23:571
2861499211,15cyclictest0-21swapper/110:30:291
2861499211,15cyclictest0-21swapper/109:12:291
2861499211,15cyclictest0-21swapper/107:28:421
2861499211,13cyclictest7280-21kworker/u8:111:45:441
2861499211,13cyclictest29167-21kworker/u8:212:12:051
2861499211,13cyclictest25449-21kworker/u8:209:21:421
2861499211,11cyclictest4454-21cpuspeed_turbos08:40:131
2861499210,6cyclictest0-21swapper/111:15:311
2861499210,2cyclictest0-21swapper/109:58:371
2861499210,15cyclictest0-21swapper/108:20:051
2860899211,11cyclictest26261-21kworker/0:007:21:350
2860899211,10cyclictest5194-21kworker/0:208:45:180
2862499206,3cyclictest0-21swapper/307:51:523
2862499203,3cyclictest86950irq/126-enp1s0-11:35:263
2862499201,6cyclictest0-21swapper/309:03:343
2862499201,6cyclictest0-21swapper/309:03:333
2862499201,3cyclictest0-21swapper/308:40:263
2862499201,3cyclictest0-21swapper/307:34:403
2861999208,3cyclictest11168-21kworker/2:107:44:222
2861999201,5cyclictest13247-21ntp_states11:25:212
2861999201,5cyclictest0-21swapper/211:58:382
2861499203,3cyclictest0-21swapper/111:07:241
28614992013,2cyclictest0-21swapper/107:31:381
28614992012,3cyclictest0-21swapper/109:33:511
2861499201,18cyclictest0-21swapper/110:00:201
2861499201,14cyclictest0-21swapper/111:34:081
2861499201,14cyclictest0-21swapper/109:50:341
2861499201,14cyclictest0-21swapper/108:57:591
2861499201,14cyclictest0-21swapper/108:45:371
2861499201,14cyclictest0-21swapper/108:31:561
2861499201,14cyclictest0-21swapper/108:10:031
2861499201,14cyclictest0-21swapper/107:54:581
2861499201,11cyclictest26551-21processes11:55:231
2861499201,11cyclictest19353-21expr08:00:131
2861499201,11cyclictest15845-21sed10:20:021
2861499201,10cyclictest1574-21cpu07:20:111
2861499200,6cyclictest0-21swapper/111:02:431
2861499200,5cyclictest0-21swapper/110:08:531
2861499200,4cyclictest0-21swapper/110:56:461
2862499199,3cyclictest878-21diskstats09:45:133
2862499198,7cyclictest0-21swapper/307:10:113
2862499196,6cyclictest751-21rngd07:29:163
2862499196,3cyclictest25189-21ntpq09:25:213
2862499193,3cyclictest0-21swapper/310:43:473
2862499193,2cyclictest0-21swapper/307:42:363
28624991914,3cyclictest0-21swapper/308:34:543
2862499190,3cyclictest0-21swapper/308:55:263
2862499190,3cyclictest0-21swapper/307:57:483
2862499190,18cyclictest0-21swapper/308:15:163
2861999198,3cyclictest0-21swapper/209:40:202
2861999192,4cyclictest859-21df09:45:132
28619991911,6cyclictest21073-21kworker/2:009:10:552
2861499191,6cyclictest0-21swapper/111:50:251
2861499191,5cyclictest0-21swapper/112:25:201
2861499191,10cyclictest10093-21latency_hist07:40:011
2861499190,1cyclictest0-21swapper/109:00:381
2861499190,1cyclictest0-21swapper/109:00:381
2861499190,14cyclictest0-21swapper/109:08:101
144362190,2sleep30-21swapper/307:49:313
2862499181,5cyclictest31703-21/usr/sbin/munin07:15:143
2862499181,5cyclictest3145-21sshd07:20:293
2862499181,3cyclictest0-21swapper/308:22:083
28624991811,4cyclictest28234-21hddtemp_smartct12:00:163
28624991811,4cyclictest20213-21/usr/sbin/munin08:00:243
2861999181,5cyclictest22312-21strings08:05:242
2860899181,6cyclictest26591-21latency08:15:190
2862499160,3cyclictest0-21swapper/308:50:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional