You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 23:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Wed Jun 17, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48262237159,25sleep00-21swapper/007:08:120
47342230156,23sleep10-21swapper/107:07:001
46372230153,24sleep20-21swapper/207:05:432
48682202166,24sleep30-21swapper/307:08:393
2072521300,5sleep1506399cyclictest10:10:111
2072521300,5sleep1506399cyclictest10:10:111
155472580,3sleep0505799cyclictest11:10:230
67012550,4sleep2507099cyclictest10:50:212
291462530,4sleep081ktimersoftd/010:30:020
507599349,4cyclictest0-21swapper/310:40:243
507599349,3cyclictest87350irq/126-enp1s0-11:20:143
507599331,4cyclictest0-21swapper/312:08:173
507599321,12cyclictest22814-21sed07:45:253
507599301,11cyclictest808-21snmpd07:25:173
506399309,3cyclictest32677-21idleruntime11:50:161
507599291,13cyclictest0-21swapper/310:20:133
506399292,25cyclictest0-21swapper/110:09:571
506399291,26cyclictest0-21swapper/110:23:241
506399291,26cyclictest0-21swapper/108:32:331
507599281,5cyclictest30135-21users09:15:243
5075992811,3cyclictest6667-21ntpq10:50:213
5075992811,3cyclictest16189-21timerwakeupswit07:30:253
507099281,6cyclictest0-21swapper/210:25:072
506399280,5cyclictest0-21swapper/109:29:061
506399280,26cyclictest0-21swapper/110:28:351
505799281,25cyclictest0-21swapper/007:48:330
507599278,11cyclictest0-21swapper/310:14:473
507599278,11cyclictest0-21swapper/310:14:473
5075992714,8cyclictest0-21swapper/309:37:253
507599271,10cyclictest7817-21head09:40:133
5075992710,4cyclictest0-21swapper/309:30:113
507099271,5cyclictest6813-21kworker/u8:210:10:162
507099271,5cyclictest6813-21kworker/u8:210:10:162
506399271,4cyclictest3315-21head08:15:221
506399271,24cyclictest0-21swapper/111:36:481
506399271,24cyclictest0-21swapper/110:53:071
506399271,24cyclictest0-21swapper/109:34:351
506399271,24cyclictest0-21swapper/107:31:551
506399270,5cyclictest0-21swapper/109:16:041
506399270,25cyclictest0-21swapper/108:40:421
507599269,3cyclictest0-21swapper/312:11:233
5075992611,3cyclictest808-21snmpd08:41:193
507599261,12cyclictest0-21swapper/311:43:313
5075992611,2cyclictest0-21swapper/311:25:113
5075992610,3cyclictest0-21swapper/308:08:393
507599260,11cyclictest0-21swapper/310:08:553
507099266,11cyclictest20828-21sort12:35:262
507099266,11cyclictest20828-21sort12:35:262
507099261,6cyclictest0-21swapper/211:35:182
506399261,23cyclictest0-21swapper/112:39:561
506399261,23cyclictest0-21swapper/112:39:561
506399261,21cyclictest700-21irqstats08:10:191
506399260,5cyclictest0-21swapper/112:29:391
506399260,5cyclictest0-21swapper/110:19:401
506399260,5cyclictest0-21swapper/109:38:371
506399260,3cyclictest0-21swapper/109:56:081
506399260,24cyclictest0-21swapper/108:53:371
5057992612,9cyclictest0-21swapper/009:25:180
505799261,22cyclictest26536-21sshd07:55:170
505799261,12cyclictest4753-21munin-run09:35:000
505799260,23cyclictest18781-21grep10:05:160
507599251,13cyclictest0-21swapper/311:07:233
507599251,13cyclictest0-21swapper/310:55:333
507599251,13cyclictest0-21swapper/310:19:513
507599251,13cyclictest0-21swapper/309:56:273
507599251,13cyclictest0-21swapper/308:13:433
507599251,12cyclictest0-21swapper/311:46:033
507599251,12cyclictest0-21swapper/311:36:553
5075992510,3cyclictest0-21swapper/312:17:513
5075992510,2cyclictest0-21swapper/311:52:553
507599250,13cyclictest0-21swapper/312:04:593
507099251,2cyclictest0-21swapper/209:11:422
5070992511,3cyclictest827-21runrttasks10:17:502
5070992510,7cyclictest87650irq/129-enp1s0-11:23:542
506399251,22cyclictest0-21swapper/107:48:271
506399250,5cyclictest0-21swapper/112:32:591
506399250,5cyclictest0-21swapper/111:41:011
506399250,5cyclictest0-21swapper/111:06:361
506399250,5cyclictest0-21swapper/110:45:581
506399250,24cyclictest0-21swapper/107:20:361
506399250,18cyclictest0-21swapper/112:03:061
505799253,10cyclictest87550irq/128-enp1s0-12:36:290
505799253,10cyclictest87550irq/128-enp1s0-12:36:290
507599249,9cyclictest0-21swapper/311:55:073
507599249,9cyclictest0-21swapper/310:00:073
507599249,3cyclictest0-21swapper/309:06:193
507599249,3cyclictest0-21swapper/307:55:433
507599248,3cyclictest0-21swapper/310:32:433
507599248,10cyclictest0-21swapper/310:38:513
507599247,3cyclictest0-21swapper/309:20:513
507599247,3cyclictest0-21swapper/308:36:353
507599241,12cyclictest0-21swapper/312:37:273
507599241,12cyclictest0-21swapper/312:37:273
507599241,10cyclictest0-21swapper/312:27:073
5075992410,3cyclictest0-21swapper/308:15:553
507599240,10cyclictest0-21swapper/308:22:593
507099241,8cyclictest0-21swapper/209:16:592
507099241,6cyclictest0-21swapper/211:33:522
507099241,6cyclictest0-21swapper/211:15:052
507099241,3cyclictest0-21swapper/212:25:162
5070992411,7cyclictest18245-1kworker/u9:212:32:392
507099241,13cyclictest0-21swapper/211:58:372
507099241,13cyclictest0-21swapper/210:24:252
506399249,3cyclictest18128-21cat10:05:011
506399242,20cyclictest0-21swapper/111:20:221
506399241,4cyclictest2890-21ntp_states11:55:221
506399240,5cyclictest0-21swapper/111:15:251
506399240,3cyclictest0-21swapper/112:20:101
506399240,3cyclictest0-21swapper/109:49:081
5057992412,2cyclictest30543-21latency_hist09:20:010
505799241,22cyclictest0-21swapper/008:51:180
505799241,20cyclictest7796-21/usr/sbin/munin10:55:150
505799241,20cyclictest25012-21sshd11:33:180
505799240,22cyclictest0-21swapper/012:15:340
252722241,19sleep225273-21sshd07:52:172
252722241,19sleep225273-21sshd07:52:172
507599239,3cyclictest0-21swapper/307:41:233
507599239,2cyclictest808-21snmpd09:02:463
507599238,3cyclictest0-21swapper/309:54:193
507599238,3cyclictest0-21swapper/307:21:393
507599237,4cyclictest0-21swapper/309:10:393
507599236,3cyclictest0-21swapper/309:27:393
507599231,9cyclictest0-21swapper/309:45:353
507599231,8cyclictest0-21swapper/308:50:183
507599231,12cyclictest0-21swapper/311:15:463
507599231,11cyclictest23527-21/usr/sbin/munin07:50:153
507599231,11cyclictest23527-21/usr/sbin/munin07:50:153
507599231,11cyclictest0-21swapper/307:15:173
507599231,10cyclictest0-21swapper/311:00:113
5075992310,3cyclictest0-21swapper/312:23:153
507599230,12cyclictest0-21swapper/311:12:033
507099236,9cyclictest755-21rngd10:02:462
507099234,10cyclictest0-21swapper/212:22:052
507099232,8cyclictest0-21swapper/212:00:182
507099232,6cyclictest0-21swapper/211:28:202
507099232,11cyclictest12222-21smart_sda08:35:232
507099232,10cyclictest0-21swapper/212:12:352
507099231,6cyclictest0-21swapper/209:55:372
507099231,4cyclictest0-21swapper/210:43:212
507099231,12cyclictest0-21swapper/211:46:402
506399231,3cyclictest6108-21latency07:10:181
506399231,2cyclictest15693-21df08:45:161
506399231,19cyclictest20062-21expr08:55:111
506399231,19cyclictest17284-21diskstats07:35:151
506399231,16cyclictest0-21swapper/110:58:551
506399231,13cyclictest0-21swapper/111:32:461
506399230,6cyclictest0-21swapper/111:49:401
506399230,5cyclictest0-21swapper/112:10:571
506399230,5cyclictest0-21swapper/111:27:031
506399230,5cyclictest0-21swapper/110:42:101
506399230,3cyclictest0-21swapper/112:05:491
505799239,3cyclictest28139-21perl11:40:140
5057992313,2cyclictest0-21swapper/009:20:260
5057992312,2cyclictest0-21swapper/011:25:110
505799231,20cyclictest0-21swapper/012:30:100
505799231,20cyclictest0-21swapper/009:35:160
5057992311,3cyclictest0-21swapper/009:55:160
505799230,21cyclictest0-21swapper/012:20:150
505799230,21cyclictest0-21swapper/007:35:160
505799230,21cyclictest0-21swapper/007:20:120
505799230,20cyclictest0-21swapper/007:15:160
202502230,5sleep3507599cyclictest08:55:123
507599229,3cyclictest0-21swapper/312:34:593
507599228,3cyclictest0-21swapper/310:45:193
507599228,3cyclictest0-21swapper/310:27:513
507599228,3cyclictest0-21swapper/308:27:153
507599228,3cyclictest0-21swapper/308:01:193
507599227,3cyclictest0-21swapper/307:11:223
507099228,3cyclictest30-21rcuc/211:05:252
507099227,3cyclictest9165-21idleruntime-cro09:45:022
507099225,4cyclictest3462-21switchtime08:15:232
507099225,4cyclictest17816-21cat08:50:122
507099224,6cyclictest30-21rcuc/209:35:162
507099223,9cyclictest19248-21ntp_states10:05:222
507099223,4cyclictest0-21swapper/210:37:222
507099222,6cyclictest5448-21ntp_states08:20:222
507099222,11cyclictest546-21hddtemp_smartct08:10:172
507099221,7cyclictest8768-21missed_timers10:55:162
507099221,7cyclictest2018-21cat09:25:242
507099221,7cyclictest10356-21users08:30:272
507099221,7cyclictest0-21swapper/209:24:122
507099221,5cyclictest16944-21users08:45:282
507099221,3cyclictest0-21swapper/211:42:472
507099221,13cyclictest0-21swapper/209:46:222
507099221,12cyclictest0-21swapper/211:00:592
507099221,11cyclictest0-21swapper/209:34:142
506399229,3cyclictest21988-21cpu09:00:131
506399225,3cyclictest87550irq/128-enp1s0-08:25:201
5063992215,4cyclictest13041-21grep09:50:251
506399221,3cyclictest12213-21users12:15:261
506399221,19cyclictest0-21swapper/110:37:381
506399221,18cyclictest0-21swapper/109:40:101
506399221,10cyclictest27396-21ntpq09:10:201
506399220,3cyclictest0-21swapper/111:03:081
506399220,3cyclictest0-21swapper/109:20:051
505799229,8cyclictest0-21swapper/009:51:260
505799227,11cyclictest0-21swapper/010:41:330
505799223,2cyclictest3160-21strings08:15:190
505799222,17cyclictest30755-21users10:30:260
505799221,5cyclictest9336-21sa112:10:170
505799221,19cyclictest0-21swapper/012:05:150
505799221,19cyclictest0-21swapper/011:15:260
505799221,19cyclictest0-21swapper/009:45:250
5057992211,6cyclictest0-21swapper/010:20:150
505799221,13cyclictest0-21swapper/012:02:360
505799221,13cyclictest0-21swapper/011:20:040
505799221,12cyclictest0-21swapper/012:27:150
505799221,10cyclictest6022-21if_enp2s007:10:170
505799221,10cyclictest139802sort07:25:250
505799221,10cyclictest12228-21cpu11:05:120
505799220,20cyclictest0-21swapper/008:30:260
505799220,20cyclictest0-21swapper/008:00:170
143302221,17sleep014448-21which08:40:590
507599218,3cyclictest0-21swapper/311:31:433
5075992112,2cyclictest18915-21awk07:40:013
507099216,4cyclictest18900-21latency_hist07:40:012
507099211,7cyclictest19863-21tr08:55:112
507099211,7cyclictest16774-21sort07:35:012
507099211,5cyclictest0-21swapper/212:19:122
507099211,3cyclictest0-21swapper/211:51:432
507099211,3cyclictest0-21swapper/209:53:492
507099211,11cyclictest7383-21irqstats08:25:192
507099211,11cyclictest30296-21grep10:30:222
507099211,11cyclictest19940-21irqstats07:40:172
507099211,11cyclictest15473-21ntp_states11:10:222
507099211,11cyclictest0-21swapper/212:07:252
507099210,9cyclictest0-21swapper/207:20:112
506399215,3cyclictest87550irq/128-enp1s0-08:05:191
506399210,3cyclictest0-21swapper/110:30:291
505799219,5cyclictest0-21swapper/010:36:180
505799218,3cyclictest7642-21ntp_states08:25:240
505799211,18cyclictest0-21swapper/009:42:040
505799211,13cyclictest0-21swapper/010:03:150
5057992110,2cyclictest0-21swapper/009:00:220
505799210,19cyclictest0-21swapper/010:15:160
505799210,19cyclictest0-21swapper/009:10:250
505799210,19cyclictest0-21swapper/007:40:110
505799210,18cyclictest32740-21if_enp2s011:50:170
507099205,8cyclictest32-21ksoftirqd/208:43:352
507099203,9cyclictest87350irq/126-enp1s0-08:05:132
507099201,6cyclictest0-21swapper/210:46:092
507099201,5cyclictest7095-21sshd07:13:262
507099201,11cyclictest808-21snmpd08:03:162
507099201,11cyclictest808-21snmpd07:17:222
507099201,10cyclictest22272-21cstates09:00:142
506399209,5cyclictest1631-21kworker/u8:107:25:211
5063992016,3cyclictest0-21swapper/108:37:521
506399201,4cyclictest26188-21tune2fs07:55:141
506399201,18cyclictest0-21swapper/107:50:201
506399201,18cyclictest0-21swapper/107:50:201
506399200,18cyclictest0-21swapper/107:18:041
505799209,3cyclictest0-21swapper/011:55:430
505799208,2cyclictest1311-21cat08:10:260
505799206,9cyclictest0-21swapper/011:36:350
505799204,8cyclictest0-21swapper/011:00:450
5057992014,3cyclictest24492-21df_inode09:05:130
505799200,18cyclictest0-21swapper/010:50:250
33912201,2sleep00-21swapper/010:45:080
248272200,4sleep20-21swapper/209:05:162
507599191,10cyclictest15354-21/usr/sbin/munin08:45:213
507099191,7cyclictest27625-21sshd07:59:172
507099191,6cyclictest21487-21cpuspeed_turbos07:45:122
507099191,5cyclictest14537-21wc07:30:012
506399192,16cyclictest0-21swapper/108:21:121
506399191,3cyclictest25738-21users09:05:261
506399191,2cyclictest0-21swapper/108:01:031
506399191,16cyclictest0-21swapper/107:40:431
505799191,6cyclictest20390-21cpu10:10:100
505799191,6cyclictest20390-21cpu10:10:100
505799191,16cyclictest0-21swapper/008:47:440
505799191,11cyclictest0-21swapper/011:45:120
507599181,9cyclictest808-21snmpd08:32:403
506399181,15cyclictest15013-21gltestperf11:10:161
505799185,3cyclictest87550irq/128-enp1s0-08:55:210
505799181,4cyclictest15125-21fschecks_count07:30:150
5057991712,3cyclictest5189-21irqstats08:20:200
127212170,2sleep00-21swapper/008:37:300
5057991611,3cyclictest23709-21cstates07:50:110
5057991611,3cyclictest23709-21cstates07:50:110
5057991610,3cyclictest31407-21swap08:05:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional