You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 03:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Nov 14, 2025 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
174870521000,5sleep00-21swapper/021:15:010
18015042940,6sleep21753-21gnome-shell23:40:342
162399900,70rtkit-daemon0-21swapper/219:06:262
17129332880,19sleep126-21ksoftirqd/119:30:251
162399780,6rtkit-daemon0-21swapper/323:35:043
162399770,6rtkit-daemon2197014-21PK-Backend00:11:073
162399730,9rtkit-daemon1622-21rtkit-daemon21:48:082
17895762720,6sleep21358-1kworker/u9:123:10:152
170412227247,16sleep10-21swapper/119:06:151
170411327238,27sleep00-21swapper/019:06:100
170414327145,17sleep30-21swapper/319:06:213
162399680,9rtkit-daemon1785-21gmain22:38:283
162399680,8rtkit-daemon860997-21snmpd21:27:513
17282092670,7chrt0-21swapper/020:15:200
162399670,9rtkit-daemon1622-21rtkit-daemon20:15:072
162399670,5rtkit-daemon2197014-21PK-Backend20:54:183
162399670,4rtkit-daemon0-21swapper/320:27:583
162399660,9rtkit-daemon1622-21rtkit-daemon19:18:122
162399660,7rtkit-daemon860997-21snmpd21:21:343
162399660,7rtkit-daemon860997-21snmpd21:21:343
1704626996543,7cyclictest34-21ksoftirqd/220:00:252
1704626996240,6cyclictest34-21ksoftirqd/219:57:362
1704626996240,6cyclictest34-21ksoftirqd/219:57:362
162399620,8rtkit-daemon1622-21rtkit-daemon20:44:352
1704621996148,10cyclictest13-21ksoftirqd/021:00:020
162399610,8rtkit-daemon1622-21rtkit-daemon20:13:492
162399610,10rtkit-daemon1622-21rtkit-daemon19:52:042
162399610,10rtkit-daemon1622-21rtkit-daemon19:36:513
162399610,10rtkit-daemon1622-21rtkit-daemon19:36:513
17098672600,5sleep3411rcuc/319:20:413
162399590,9rtkit-daemon1622-21rtkit-daemon21:04:103
162399590,9rtkit-daemon1622-21rtkit-daemon19:11:592
162399590,8rtkit-daemon1622-21rtkit-daemon19:27:182
162399580,9rtkit-daemon1622-21rtkit-daemon23:43:160
162399580,9rtkit-daemon1622-21rtkit-daemon23:37:080
162399580,42rtkit-daemon34-21ksoftirqd/223:30:022
162399570,9rtkit-daemon1622-21rtkit-daemon20:56:413
162399570,9rtkit-daemon1622-21rtkit-daemon19:51:423
162399560,9rtkit-daemon1622-21rtkit-daemon22:41:553
162399560,8rtkit-daemon1622-21rtkit-daemon22:49:213
162399560,8rtkit-daemon1622-21rtkit-daemon22:49:213
162399550,9rtkit-daemon1622-21rtkit-daemon22:53:313
162399550,9rtkit-daemon1622-21rtkit-daemon22:38:082
162399550,9rtkit-daemon1622-21rtkit-daemon20:08:173
162399550,8rtkit-daemon1622-21rtkit-daemon19:26:223
162399540,9rtkit-daemon1622-21rtkit-daemon22:27:333
162399540,9rtkit-daemon1622-21rtkit-daemon21:54:262
162399540,27rtkit-daemon0-21swapper/022:27:120
162399530,8rtkit-daemon1622-21rtkit-daemon22:15:252
162399530,7rtkit-daemon1792038-21python323:15:303
1704631995128,19cyclictest1753-21gnome-shell00:35:273
1704631995128,19cyclictest1753-21gnome-shell00:35:263
1704631995026,20cyclictest1753-21gnome-shell20:32:313
1704631995025,20cyclictest1753-21gnome-shell23:30:173
1704626995033,6cyclictest34-21ksoftirqd/223:05:242
1704631994925,20cyclictest1753-21gnome-shell22:59:383
1704626994924,15cyclictest34-21ksoftirqd/200:25:202
1704626994825,14cyclictest34-21ksoftirqd/221:30:322
1704626994825,14cyclictest34-21ksoftirqd/200:30:312
17691012470,5sleep1251rcuc/122:10:281
1704631994723,19cyclictest1753-21gnome-shell00:30:063
170462699476,12cyclictest141rcu_preempt21:29:232
162399470,40rtkit-daemon34-21ksoftirqd/219:35:312
162399470,40rtkit-daemon34-21ksoftirqd/219:35:312
1704631994624,18cyclictest1753-21gnome-shell22:30:273
1704626994623,14cyclictest34-21ksoftirqd/220:20:312
1704626994615,12cyclictest34-21ksoftirqd/222:40:282
1704623994618,6cyclictest26-21ksoftirqd/123:11:241
162399460,38rtkit-daemon2197014-21PK-Backend22:07:262
1704631994510,16cyclictest1753-21gnome-shell20:41:233
1704626994524,16cyclictest34-21ksoftirqd/222:30:162
1704626994523,18cyclictest34-21ksoftirqd/219:20:282
1704626994522,7cyclictest1753-21gnome-shell23:21:202
1704626994522,14cyclictest34-21ksoftirqd/222:55:402
1704621994525,8cyclictest13-21ksoftirqd/020:42:160
1704621994525,7cyclictest13-21ksoftirqd/020:04:260
162399450,36rtkit-daemon1704613-21cyclictest20:49:372
17884562440,7sleep32197014-21PK-Backend23:05:293
17727402440,8chrt1772736-21threads22:20:301
170462699448,21cyclictest34-21ksoftirqd/200:25:002
1704626994427,13cyclictest1751157-21awk21:20:212
1704626994427,13cyclictest1751157-21awk21:20:212
1704626994425,6cyclictest34-21ksoftirqd/200:00:022
1704626994425,15cyclictest34-21ksoftirqd/221:05:152
1704626994425,14cyclictest34-21ksoftirqd/223:00:282
1704626994425,14cyclictest34-21ksoftirqd/200:15:282
1704626994425,14cyclictest34-21ksoftirqd/200:15:282
1704626994422,14cyclictest34-21ksoftirqd/221:05:012
170462699441,39cyclictest1753-21gnome-shell21:36:012
1704623994415,10cyclictest1723009-21entropy20:00:171
1704623994414,25cyclictest1799928-40lsb_release23:36:271
1704621994433,7cyclictest1753-21gnome-shell22:00:150
1704621994425,7cyclictest13-21ksoftirqd/021:51:330
1704621994425,7cyclictest13-21ksoftirqd/021:35:330
1704621994425,7cyclictest13-21ksoftirqd/019:39:470
1704621994425,7cyclictest13-21ksoftirqd/019:39:470
1704621994424,7cyclictest13-21ksoftirqd/022:47:520
1704621994424,7cyclictest13-21ksoftirqd/022:47:520
170462699438,18cyclictest1753-21gnome-shell21:10:352
170462699438,18cyclictest1753-21gnome-shell20:29:122
170462699437,19cyclictest1753-21gnome-shell00:08:032
170462699436,20cyclictest1753-21gnome-shell00:02:232
1704626994325,6cyclictest34-21ksoftirqd/222:00:342
1704626994325,14cyclictest34-21ksoftirqd/223:35:252
1704626994325,14cyclictest34-21ksoftirqd/200:35:342
1704626994325,14cyclictest34-21ksoftirqd/200:35:342
1704626994324,6cyclictest34-21ksoftirqd/222:15:012
1704626994324,14cyclictest34-21ksoftirqd/222:00:022
1704626994323,12cyclictest34-21ksoftirqd/220:39:032
1704626994320,14cyclictest34-21ksoftirqd/223:45:222
1704623994327,12cyclictest1741745-21/usr/sbin/munin20:55:011
1704623994324,14cyclictest26-21ksoftirqd/120:25:331
1704623994323,15cyclictest26-21ksoftirqd/121:16:341
1704623994323,15cyclictest26-21ksoftirqd/120:10:361
1704623994321,17cyclictest1738069-20journal-offline20:42:401
1704623994317,22cyclictest26-21ksoftirqd/119:15:281
1704623994317,13cyclictest26-21ksoftirqd/119:57:091
1704623994317,13cyclictest26-21ksoftirqd/119:57:091
1704621994332,7cyclictest1753-21gnome-shell19:33:300
1704621994331,8cyclictest1753-21gnome-shell23:34:590
1704621994331,8cyclictest1753-21gnome-shell00:17:120
1704621994331,8cyclictest1753-21gnome-shell00:17:110
1704621994324,7cyclictest13-21ksoftirqd/020:52:250
1704621994324,7cyclictest13-21ksoftirqd/020:23:010
1704621994321,7cyclictest13-21ksoftirqd/021:34:270
1704621994320,7cyclictest13-21ksoftirqd/019:46:070
162399430,27rtkit-daemon42-21ksoftirqd/321:39:063
17183662420,5sleep2331rcuc/219:45:312
1704631994217,20cyclictest1753-21gnome-shell22:05:153
170462699423,18cyclictest1753-21gnome-shell20:55:142
1704626994225,14cyclictest34-21ksoftirqd/220:50:172
1704626994224,6cyclictest34-21ksoftirqd/223:50:492
1704626994224,14cyclictest34-21ksoftirqd/223:16:192
1704626994224,14cyclictest34-21ksoftirqd/222:51:252
1704626994224,14cyclictest34-21ksoftirqd/221:15:152
1704626994224,13cyclictest34-21ksoftirqd/221:40:142
1704626994223,14cyclictest34-21ksoftirqd/223:30:342
1704626994223,14cyclictest34-21ksoftirqd/219:44:592
170462699422,22cyclictest1753-21gnome-shell20:33:212
1704623994224,14cyclictest26-21ksoftirqd/122:15:261
1704623994224,14cyclictest26-21ksoftirqd/121:56:011
1704623994224,14cyclictest26-21ksoftirqd/121:54:361
1704623994224,14cyclictest26-21ksoftirqd/121:39:521
1704623994224,14cyclictest26-21ksoftirqd/120:56:411
1704623994222,7cyclictest26-21ksoftirqd/119:35:151
1704623994222,7cyclictest26-21ksoftirqd/119:35:151
1704623994222,17cyclictest1753-21gnome-shell00:06:141
1704623994216,22cyclictest1758380-21open_files21:40:301
1704621994230,8cyclictest1753-21gnome-shell23:15:340
1704621994230,8cyclictest1753-21gnome-shell21:05:420
1704621994228,10cyclictest1806992-21taskset23:57:590
1704621994225,13cyclictest1738517-21/usr/sbin/munin20:45:190
1704621994224,6cyclictest13-21ksoftirqd/023:04:320
1704621994224,6cyclictest13-21ksoftirqd/019:55:010
1704621994224,6cyclictest13-21ksoftirqd/000:30:040
1704621994223,7cyclictest13-21ksoftirqd/022:58:250
1704621994223,7cyclictest13-21ksoftirqd/022:39:360
1704621994223,6cyclictest13-21ksoftirqd/021:48:310
1704621994222,6cyclictest13-21ksoftirqd/022:40:140
1704621994222,5cyclictest13-21ksoftirqd/023:10:320
1704621994221,7cyclictest13-21ksoftirqd/019:57:090
1704621994221,7cyclictest13-21ksoftirqd/019:57:080
1704621994221,7cyclictest13-21ksoftirqd/019:19:070
1704621994220,7cyclictest13-21ksoftirqd/020:31:410
170462199421,21cyclictest13-21ksoftirqd/000:40:000
170462199421,21cyclictest13-21ksoftirqd/000:40:000
1704621994210,8cyclictest141rcu_preempt00:26:150
1704631994126,11cyclictest1801690-21sshd23:41:413
1704631994124,12cyclictest438-20systemd-journal23:28:103
1704631994118,19cyclictest1753-21gnome-shell19:49:413
1704631994118,19cyclictest1753-21gnome-shell00:23:263
170463199411,10cyclictest2197014-21PK-Backend00:10:023
1704626994124,12cyclictest34-21ksoftirqd/220:05:222
1704626994122,7cyclictest34-21ksoftirqd/222:25:282
1704626994120,13cyclictest34-21ksoftirqd/200:11:072
1704623994127,10cyclictest1798187-21apt-get23:35:011
1704623994126,11cyclictest1821388-21munin-plugin-st00:40:011
1704623994126,11cyclictest1821388-21munin-plugin-st00:40:011
1704623994124,6cyclictest26-21ksoftirqd/100:01:571
1704623994124,13cyclictest26-21ksoftirqd/119:28:011
1704623994123,14cyclictest26-21ksoftirqd/123:45:321
1704623994123,14cyclictest26-21ksoftirqd/123:25:371
1704623994123,14cyclictest26-21ksoftirqd/123:18:131
1704623994123,14cyclictest26-21ksoftirqd/123:04:261
1704623994123,14cyclictest26-21ksoftirqd/122:45:011
1704623994123,14cyclictest26-21ksoftirqd/122:33:481
1704623994123,14cyclictest26-21ksoftirqd/122:03:131
1704623994123,14cyclictest26-21ksoftirqd/121:25:271
1704623994123,14cyclictest26-21ksoftirqd/119:12:111
1704623994123,13cyclictest438-20systemd-journal21:13:421
1704623994123,13cyclictest26-21ksoftirqd/100:30:031
1704621994124,5cyclictest13-21ksoftirqd/022:20:130
1704621994123,7cyclictest1753-21gnome-shell22:33:120
1704621994123,6cyclictest13-21ksoftirqd/023:50:170
1704621994123,6cyclictest13-21ksoftirqd/023:06:410
1704621994123,6cyclictest13-21ksoftirqd/022:10:220
1704621994123,6cyclictest13-21ksoftirqd/019:42:230
1704621994123,6cyclictest13-21ksoftirqd/019:24:330
1704621994123,6cyclictest13-21ksoftirqd/000:10:070
1704621994122,7cyclictest13-21ksoftirqd/021:00:110
1704621994122,6cyclictest13-21ksoftirqd/020:25:230
1704621994122,6cyclictest13-21ksoftirqd/000:00:050
1704621994121,6cyclictest13-21ksoftirqd/021:18:000
1704621994120,6cyclictest13-21ksoftirqd/023:48:390
1704621994118,7cyclictest13-21ksoftirqd/000:09:320
162399410,33rtkit-daemon707415-21gmain22:20:032
1704623994029,7cyclictest1806359-21tail23:55:251
1704623994029,7cyclictest1753-21gnome-shell00:27:421
1704623994023,6cyclictest26-21ksoftirqd/121:30:141
1704623994023,13cyclictest26-21ksoftirqd/123:09:411
1704623994023,13cyclictest26-21ksoftirqd/121:23:421
1704623994023,13cyclictest26-21ksoftirqd/121:23:421
1704623994023,13cyclictest26-21ksoftirqd/121:00:321
1704623994023,13cyclictest26-21ksoftirqd/119:22:431
1704623994023,13cyclictest26-21ksoftirqd/100:22:291
1704623994022,6cyclictest26-21ksoftirqd/121:47:251
1704623994022,5cyclictest26-21ksoftirqd/121:05:131
1704623994022,14cyclictest26-21ksoftirqd/123:40:061
1704623994022,14cyclictest26-21ksoftirqd/123:20:181
1704623994022,14cyclictest26-21ksoftirqd/122:51:201
1704623994022,14cyclictest26-21ksoftirqd/119:50:311
1704623994022,14cyclictest26-21ksoftirqd/100:10:091
1704623994022,13cyclictest26-21ksoftirqd/120:35:351
1704623994021,14cyclictest26-21ksoftirqd/100:15:211
1704623994021,14cyclictest26-21ksoftirqd/100:15:211
1704623994020,15cyclictest26-21ksoftirqd/120:20:291
1704623994018,14cyclictest26-21ksoftirqd/122:06:291
1704621994029,7cyclictest1753-21gnome-shell21:25:100
1704621994028,8cyclictest1753-21gnome-shell19:12:120
1704621994023,6cyclictest13-21ksoftirqd/019:28:470
1704621994023,5cyclictest13-21ksoftirqd/020:08:440
1704621994022,7cyclictest13-21ksoftirqd/023:20:250
1704621994022,6cyclictest13-21ksoftirqd/022:50:150
1704621994022,6cyclictest13-21ksoftirqd/022:18:580
1704621994022,6cyclictest13-21ksoftirqd/022:06:050
1704621994022,6cyclictest13-21ksoftirqd/020:14:150
1704631993922,13cyclictest1817640-21pool-org.gnome.00:30:013
1704631993919,15cyclictest42-21ksoftirqd/320:35:323
1704631993915,20cyclictest1753-21gnome-shell21:50:153
1704631993913,10cyclictest855-21dbus-daemon00:00:063
1704626993922,13cyclictest34-21ksoftirqd/219:35:012
1704626993921,14cyclictest34-21ksoftirqd/222:47:492
1704626993921,14cyclictest34-21ksoftirqd/222:47:492
1704623993923,12cyclictest26-21ksoftirqd/122:45:321
1704623993923,12cyclictest26-21ksoftirqd/122:45:311
1704623993922,5cyclictest26-21ksoftirqd/122:55:241
1704623993922,5cyclictest26-21ksoftirqd/120:30:291
1704623993922,5cyclictest26-21ksoftirqd/119:41:271
1704623993922,13cyclictest26-21ksoftirqd/122:26:281
1704623993922,13cyclictest26-21ksoftirqd/120:15:171
1704623993921,13cyclictest26-21ksoftirqd/120:45:191
1704621993928,7cyclictest1753-21gnome-shell21:42:270
1704621993921,6cyclictest13-21ksoftirqd/023:26:140
1704621993921,6cyclictest13-21ksoftirqd/020:35:370
1704621993921,5cyclictest13-21ksoftirqd/021:20:240
1704621993921,5cyclictest13-21ksoftirqd/021:20:240
1704621993920,6cyclictest13-21ksoftirqd/021:58:270
1704621993918,6cyclictest13-21ksoftirqd/000:24:190
17772552380,5chrt0-21swapper/122:35:161
1704631993824,11cyclictest42-21ksoftirqd/320:00:273
170463199382,32cyclictest1753-21gnome-shell23:20:343
1704631993816,18cyclictest1764581-21cron22:00:003
1704631993813,7cyclictest1753-21gnome-shell23:14:383
170463199381,15cyclictest17543122cut21:30:163
1704623993821,6cyclictest26-21ksoftirqd/123:53:571
1704623993821,13cyclictest26-21ksoftirqd/119:45:041
1704623993820,6cyclictest26-21ksoftirqd/120:05:081
170463199372,11cyclictest1753-21gnome-shell19:18:223
170463199372,11cyclictest141rcu_preempt19:10:313
1704631993719,14cyclictest1753-21gnome-shell19:43:103
170463199367,11cyclictest1753-21gnome-shell20:20:583
1704631993615,16cyclictest42-21ksoftirqd/321:40:273
170463199357,24cyclictest1807271-21cups-browsed00:00:013
1704631993524,8cyclictest42-21ksoftirqd/321:16:083
170463199352,14cyclictest1804453-21irqstats23:50:233
1704631993518,13cyclictest1740033-21pool-org.gnome.20:50:013
162399350,6rtkit-daemon2197014-21PK-Backend23:47:493
1704631993422,8cyclictest1753-21gnome-shell00:15:413
1704631993422,8cyclictest1753-21gnome-shell00:15:403
1704631993419,11cyclictest42-21ksoftirqd/322:10:153
1704631993418,12cyclictest1786361-21hddtemp_smartct23:00:193
170463199341,19cyclictest2197014-21PK-Backend21:45:313
1704631993411,20cyclictest1721748-21processes19:55:283
1704631993411,20cyclictest1721748-21processes19:55:283
162399340,26rtkit-daemon1622-21rtkit-daemon22:21:113
170463199332,9cyclictest1753-21gnome-shell22:01:583
1704631993321,8cyclictest1753-21gnome-shell20:18:443
1704631993317,12cyclictest1713092-21sensors_temp19:30:293
170463199331,5cyclictest2197014-21PK-Backend20:14:233
1704631993312,18cyclictest42-21ksoftirqd/321:06:093
1704631993221,7cyclictest1770833-21proc_pri22:15:273
170463199321,4cyclictest2197014-21PK-Backend21:13:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional