You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-22 - 19:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Oct 22, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1623991180,8rtkit-daemon1622-21rtkit-daemon09:32:210
1623991060,8rtkit-daemon1622-21rtkit-daemon10:09:130
1623991050,9rtkit-daemon1622-21rtkit-daemon12:38:290
1623991040,9rtkit-daemon1622-21rtkit-daemon09:38:120
1623991030,8rtkit-daemon1622-21rtkit-daemon11:14:130
1623991010,10rtkit-daemon1622-21rtkit-daemon10:59:570
1623991010,10rtkit-daemon1622-21rtkit-daemon10:59:570
1623991000,11rtkit-daemon1622-21rtkit-daemon10:24:330
14325292970,5sleep31419737-21kworker/u8:108:29:353
162399950,8rtkit-daemon1622-21rtkit-daemon08:34:090
162399930,8rtkit-daemon1622-21rtkit-daemon08:24:560
162399890,9rtkit-daemon1622-21rtkit-daemon09:35:221
140408299877,13cyclictest34-21ksoftirqd/210:30:292
162399860,8rtkit-daemon1622-21rtkit-daemon07:52:510
140407699868,8cyclictest141rcu_preempt10:30:060
162399840,9rtkit-daemon1622-21rtkit-daemon09:17:341
162399830,8rtkit-daemon1622-21rtkit-daemon08:25:330
140359928253,20sleep30-21swapper/307:06:013
140407999791,16cyclictest26-21ksoftirqd/110:32:261
14247832780,14sleep3421ksoftirqd/308:05:243
14247832780,14sleep3421ksoftirqd/308:05:243
14720522770,6sleep20-21swapper/210:20:142
1404086997614,43cyclictest141rcu_preempt10:26:173
1404076997620,49cyclictest13-21ksoftirqd/010:28:160
1404082997515,8cyclictest17531gnome-shell10:26:542
162399740,6rtkit-daemon1622-21rtkit-daemon11:51:471
140363527347,16sleep20-21swapper/207:06:282
1404086997257,11cyclictest42-21ksoftirqd/310:32:543
1404079997213,28cyclictest26-21ksoftirqd/109:05:011
140364327155,9sleep00-21swapper/007:06:340
14954452690,4sleep22197014-21PK-Backend11:25:232
14854032690,5sleep22197014-21PK-Backend10:57:422
14854032690,5sleep22197014-21PK-Backend10:57:422
140408699692,11cyclictest0-21swapper/310:24:203
14513792680,5chrt854-21avahi-daemon09:20:353
140407999682,13cyclictest0-21swapper/110:26:341
15101852670,6chrt0-21swapper/312:05:333
162399660,8rtkit-daemon1622-21rtkit-daemon08:03:290
140377326438,19sleep10-21swapper/107:08:151
140407999631,14cyclictest26-21ksoftirqd/111:44:061
162399620,8rtkit-daemon1622-21rtkit-daemon10:12:250
140407999619,11cyclictest26-21ksoftirqd/108:35:241
1404076996120,9cyclictest13-21ksoftirqd/010:40:130
1404079996027,8cyclictest26-21ksoftirqd/111:02:191
1404079996027,8cyclictest26-21ksoftirqd/107:45:541
1404079996027,10cyclictest26-21ksoftirqd/112:13:561
1404082995927,10cyclictest34-21ksoftirqd/207:35:022
1404079995930,8cyclictest26-21ksoftirqd/112:02:481
1404079995930,8cyclictest26-21ksoftirqd/112:02:481
1404079995926,8cyclictest26-21ksoftirqd/110:40:161
1404079995917,18cyclictest26-21ksoftirqd/111:37:111
1404079995914,18cyclictest26-21ksoftirqd/108:45:011
140407999591,20cyclictest26-21ksoftirqd/110:50:281
1404076995935,9cyclictest13-21ksoftirqd/011:10:000
140407699590,55cyclictest1622-21rtkit-daemon08:54:390
15188732580,15chrt13-21ksoftirqd/012:30:340
15157612580,7sleep21515758-21dump-pmu-power12:25:012
15157612580,7sleep21515758-21dump-pmu-power12:25:002
1404079995815,18cyclictest26-21ksoftirqd/107:40:011
1404079995814,19cyclictest26-21ksoftirqd/110:15:001
1404079995811,20cyclictest26-21ksoftirqd/109:45:011
1404076995835,13cyclictest13-21ksoftirqd/008:40:300
1404076995833,9cyclictest13-21ksoftirqd/007:35:010
1404076995821,20cyclictest13-21ksoftirqd/009:25:000
162399570,9rtkit-daemon1622-21rtkit-daemon10:24:191
1404079995731,8cyclictest26-21ksoftirqd/109:15:001
1404079995714,19cyclictest26-21ksoftirqd/111:30:181
140407999571,23cyclictest1416997-21awk07:45:001
1404079995711,20cyclictest26-21ksoftirqd/111:05:131
1404076995730,9cyclictest13-21ksoftirqd/007:20:010
1404076995711,18cyclictest1486354-21irqstats11:00:210
162399560,8rtkit-daemon1622-21rtkit-daemon10:01:520
162399560,8rtkit-daemon1622-21rtkit-daemon08:26:071
140407999563,19cyclictest26-21ksoftirqd/108:15:151
1404079995616,8cyclictest26-21ksoftirqd/109:24:301
140407999561,14cyclictest26-21ksoftirqd/108:25:011
1404079995611,21cyclictest26-21ksoftirqd/110:45:241
1404079995611,17cyclictest26-21ksoftirqd/112:20:271
1404079995611,17cyclictest26-21ksoftirqd/112:20:271
1404076995623,8cyclictest1069-21runrttasks10:48:270
1404076995611,15cyclictest13-21ksoftirqd/009:05:160
1404086995530,21cyclictest1753-21gnome-shell09:55:253
140407999558,21cyclictest26-21ksoftirqd/109:25:191
140407999556,20cyclictest1518525-21sh12:30:271
1404079995513,18cyclictest26-21ksoftirqd/111:21:131
1404076995524,10cyclictest13-21ksoftirqd/011:44:060
1404076995520,16cyclictest13-21ksoftirqd/007:20:180
1404076995518,9cyclictest13-21ksoftirqd/007:45:020
1404076995517,11cyclictest13-21ksoftirqd/007:36:230
1404076995516,17cyclictest13-21ksoftirqd/007:55:290
140408699549,27cyclictest15057652tr11:55:123
1404082995418,20cyclictest34-21ksoftirqd/208:05:002
1404079995422,10cyclictest26-21ksoftirqd/112:09:591
1404079995416,18cyclictest26-21ksoftirqd/111:20:011
140407999541,12cyclictest26-21ksoftirqd/111:30:001
1404079995410,11cyclictest26-21ksoftirqd/108:30:231
1404076995432,9cyclictest13-21ksoftirqd/007:50:010
1404076995430,8cyclictest13-21ksoftirqd/007:15:010
1404076995428,9cyclictest1502604-21taskset11:45:220
1404076995420,16cyclictest13-21ksoftirqd/008:35:150
1404076995415,18cyclictest13-21ksoftirqd/012:25:200
1404076995411,26cyclictest13-21ksoftirqd/009:40:160
140407999535,14cyclictest0-21swapper/112:29:261
140407999531,17cyclictest26-21ksoftirqd/108:45:301
140407999531,12cyclictest26-21ksoftirqd/108:59:521
1404076995327,9cyclictest13-21ksoftirqd/009:00:000
1404076995326,9cyclictest13-21ksoftirqd/011:37:110
1404076995323,9cyclictest13-21ksoftirqd/012:10:010
1404076995318,23cyclictest13-21ksoftirqd/012:01:530
1404076995318,23cyclictest13-21ksoftirqd/012:01:530
1404076995313,18cyclictest13-21ksoftirqd/007:25:150
140407699531,14cyclictest1506116-21/usr/sbin/munin11:55:190
1404086995241,8cyclictest42-21ksoftirqd/309:35:023
1404082995230,14cyclictest34-21ksoftirqd/208:42:172
140407999521,13cyclictest26-21ksoftirqd/110:00:011
140407999521,13cyclictest26-21ksoftirqd/108:00:001
140407999521,12cyclictest26-21ksoftirqd/111:45:281
1404076995225,9cyclictest13-21ksoftirqd/010:40:010
1404076995224,9cyclictest13-21ksoftirqd/011:30:210
1404076995221,15cyclictest13-21ksoftirqd/008:15:010
1404076995215,18cyclictest13-21ksoftirqd/008:15:140
162399510,8rtkit-daemon1622-21rtkit-daemon10:16:100
1404079995120,13cyclictest1506232-21sshd11:55:271
1404079995110,19cyclictest26-21ksoftirqd/110:35:211
1404079995110,11cyclictest26-21ksoftirqd/107:20:251
1404076995113,17cyclictest13-21ksoftirqd/012:15:200
1404082995015,21cyclictest34-21ksoftirqd/208:20:292
140408299501,12cyclictest1425680-21chrt08:10:022
140408299501,12cyclictest1425680-21chrt08:10:022
140407999506,10cyclictest26-21ksoftirqd/109:45:191
1404079995018,9cyclictest26-21ksoftirqd/107:15:251
140407999500,44cyclictest1622-21rtkit-daemon10:09:141
140407699502,19cyclictest1460206-21df09:45:150
1404076995017,9cyclictest13-21ksoftirqd/011:20:050
1404076995015,17cyclictest13-21ksoftirqd/011:15:300
1404076995013,7cyclictest13-21ksoftirqd/008:05:220
1404076995013,7cyclictest13-21ksoftirqd/008:05:220
1404076995013,21cyclictest13-21ksoftirqd/012:15:010
1404076995013,19cyclictest13-21ksoftirqd/011:25:230
1404082994916,12cyclictest34-21ksoftirqd/211:20:042
1404076994928,16cyclictest13-21ksoftirqd/009:00:140
1404076994917,11cyclictest13-21ksoftirqd/009:15:010
1404076994916,10cyclictest13-21ksoftirqd/010:00:020
140408299486,13cyclictest34-21ksoftirqd/212:05:192
140408299485,14cyclictest1753-21gnome-shell07:10:252
1404082994824,15cyclictest34-21ksoftirqd/209:42:252
1404082994818,9cyclictest34-21ksoftirqd/209:35:022
1404082994816,6cyclictest141rcu_preempt10:10:232
1404082994812,10cyclictest34-21ksoftirqd/211:35:012
1404082994811,5cyclictest141rcu_preempt07:40:272
140407999486,10cyclictest26-21ksoftirqd/112:40:001
140407999483,19cyclictest26-21ksoftirqd/111:00:011
140407999483,19cyclictest26-21ksoftirqd/111:00:011
140407999482,15cyclictest26-21ksoftirqd/108:10:221
140407999481,19cyclictest26-21ksoftirqd/107:25:141
140407999481,12cyclictest1455342-21taskset09:33:191
1404076994817,16cyclictest13-21ksoftirqd/009:29:210
1404076994810,5cyclictest141rcu_preempt10:55:010
14304722470,8sleep30-21swapper/308:20:333
140408299477,6cyclictest141rcu_preempt07:25:002
1404082994725,14cyclictest34-21ksoftirqd/209:52:272
1404082994719,24cyclictest34-21ksoftirqd/209:55:182
1404082994714,8cyclictest34-21ksoftirqd/207:52:082
1404082994714,6cyclictest141rcu_preempt08:50:182
1404082994711,9cyclictest34-21ksoftirqd/209:00:152
140408299471,14cyclictest1500443-21sshd11:40:182
140407999471,18cyclictest26-21ksoftirqd/108:00:331
1404076994712,17cyclictest13-21ksoftirqd/011:53:430
1404086994624,18cyclictest860997-21snmpd09:53:153
1404086994624,14cyclictest42-21ksoftirqd/307:42:283
1404086994622,7cyclictest1753-21gnome-shell08:15:333
1404082994629,6cyclictest34-21ksoftirqd/208:26:342
1404082994624,13cyclictest34-21ksoftirqd/211:50:302
1404082994623,15cyclictest34-21ksoftirqd/210:02:242
1404082994623,14cyclictest34-21ksoftirqd/211:02:142
1404082994623,14cyclictest34-21ksoftirqd/208:55:512
1404082994623,14cyclictest34-21ksoftirqd/207:55:122
1404082994622,15cyclictest34-21ksoftirqd/209:49:282
1404082994619,23cyclictest34-21ksoftirqd/212:00:142
1404082994619,23cyclictest34-21ksoftirqd/212:00:142
1404082994616,8cyclictest34-21ksoftirqd/209:35:142
1404082994612,6cyclictest141rcu_preempt07:35:102
1404082994611,9cyclictest34-21ksoftirqd/209:05:332
140408299461,13cyclictest1448020-21chrt09:10:312
1404082994611,17cyclictest1502518-21entropy11:45:202
140407999467,17cyclictest26-21ksoftirqd/107:15:011
140407999466,5cyclictest141rcu_preempt07:55:011
140407999464,8cyclictest26-21ksoftirqd/108:10:001
140407999464,8cyclictest26-21ksoftirqd/108:10:001
140407999461,14cyclictest26-21ksoftirqd/112:15:161
140407699469,15cyclictest13-21ksoftirqd/012:20:200
140407699469,15cyclictest13-21ksoftirqd/012:20:200
1404076994615,18cyclictest13-21ksoftirqd/009:19:120
1404076994612,18cyclictest13-21ksoftirqd/009:51:190
1404076994612,16cyclictest13-21ksoftirqd/008:45:310
15168982450,8chrt2197014-21PK-Backend12:25:292
140408699457,20cyclictest0-21swapper/310:20:013
1404086994523,13cyclictest42-21ksoftirqd/310:10:003
1404086994522,14cyclictest42-21ksoftirqd/308:42:283
1404086994519,10cyclictest1753-21gnome-shell11:10:513
1404086994517,11cyclictest42-21ksoftirqd/311:37:113
1404086994517,10cyclictest1449035-21cat09:15:143
1404086994514,11cyclictest42-21ksoftirqd/312:20:183
1404086994514,11cyclictest42-21ksoftirqd/312:20:183
1404086994512,19cyclictest42-21ksoftirqd/310:38:283
1404082994524,6cyclictest34-21ksoftirqd/211:35:332
1404082994524,13cyclictest34-21ksoftirqd/211:15:132
1404082994523,13cyclictest34-21ksoftirqd/212:15:392
1404082994523,13cyclictest34-21ksoftirqd/211:59:312
1404082994523,13cyclictest34-21ksoftirqd/210:06:232
1404082994523,13cyclictest34-21ksoftirqd/208:10:142
1404082994522,6cyclictest34-21ksoftirqd/211:05:232
1404082994522,16cyclictest34-21ksoftirqd/209:27:402
1404082994522,14cyclictest34-21ksoftirqd/207:18:592
140408299451,16cyclictest1489546-21cstates11:10:132
1404082994511,25cyclictest34-21ksoftirqd/212:35:002
140407999456,8cyclictest26-21ksoftirqd/107:35:021
140407999455,10cyclictest26-21ksoftirqd/110:15:141
14925642440,6chrt2197014-21PK-Backend11:20:013
140408699448,17cyclictest1753-21gnome-shell12:26:053
140408699444,21cyclictest42-21ksoftirqd/312:39:593
1404086994424,12cyclictest42-21ksoftirqd/311:42:443
1404086994422,14cyclictest42-21ksoftirqd/312:00:203
1404086994422,14cyclictest42-21ksoftirqd/312:00:203
1404086994420,15cyclictest42-21ksoftirqd/307:36:133
140408299444,16cyclictest34-21ksoftirqd/210:15:242
1404082994424,16cyclictest34-21ksoftirqd/210:37:582
1404082994424,16cyclictest34-21ksoftirqd/207:26:232
1404082994423,13cyclictest34-21ksoftirqd/210:47:212
1404082994423,13cyclictest34-21ksoftirqd/208:37:282
1404082994422,14cyclictest34-21ksoftirqd/209:21:182
1404082994422,13cyclictest34-21ksoftirqd/210:43:112
140408299441,15cyclictest1449637-21/usr/sbin/munin09:15:282
1404082994410,5cyclictest141rcu_preempt08:15:132
140407999446,5cyclictest141rcu_preempt11:15:011
140407999441,14cyclictest1465501-21if_err_enp2s010:00:221
14175502430,7sleep22197014-21PK-Backend07:45:142
1404086994332,7cyclictest1753-21gnome-shell10:00:233
1404086994326,13cyclictest42-21ksoftirqd/310:55:223
1404086994326,13cyclictest42-21ksoftirqd/310:55:223
1404086994324,14cyclictest42-21ksoftirqd/311:00:243
1404086994322,13cyclictest42-21ksoftirqd/310:50:293
1404086994322,12cyclictest42-21ksoftirqd/311:28:423
1404086994321,13cyclictest42-21ksoftirqd/309:42:003
1404086994315,11cyclictest42-21ksoftirqd/308:30:303
1404082994322,13cyclictest34-21ksoftirqd/210:50:312
1404082994322,12cyclictest34-21ksoftirqd/212:38:412
1404082994322,12cyclictest34-21ksoftirqd/212:10:142
140407999431,22cyclictest26-21ksoftirqd/109:50:281
140407999431,12cyclictest26-21ksoftirqd/109:05:311
140408699427,12cyclictest42-21ksoftirqd/308:48:403
1404086994231,7cyclictest1753-21gnome-shell07:13:063
1404086994225,6cyclictest42-21ksoftirqd/309:00:533
1404086994225,13cyclictest42-21ksoftirqd/311:50:223
1404086994224,14cyclictest42-21ksoftirqd/307:25:183
1404086994223,14cyclictest42-21ksoftirqd/307:15:263
1404086994222,13cyclictest42-21ksoftirqd/308:57:073
1404086994219,14cyclictest42-21ksoftirqd/309:49:103
1404082994221,6cyclictest34-21ksoftirqd/208:30:092
140407999426,10cyclictest26-21ksoftirqd/108:50:291
14982232410,5sleep32197014-21PK-Backend11:35:013
140408699419,19cyclictest141rcu_preempt09:35:293
1404086994123,6cyclictest42-21ksoftirqd/308:00:313
1404086994123,14cyclictest42-21ksoftirqd/307:50:263
1404086994123,13cyclictest42-21ksoftirqd/312:30:243
1404086994123,13cyclictest42-21ksoftirqd/311:06:023
1404086994122,11cyclictest42-21ksoftirqd/310:48:313
1404086994121,11cyclictest42-21ksoftirqd/311:20:163
1404086994120,17cyclictest42-21ksoftirqd/308:39:123
1404086994119,13cyclictest42-21ksoftirqd/309:10:143
1404086994118,7cyclictest1753-21gnome-shell12:19:513
1404082994122,5cyclictest34-21ksoftirqd/208:45:492
1404086994023,6cyclictest42-21ksoftirqd/309:26:383
1404086994022,14cyclictest42-21ksoftirqd/309:10:013
1404086994022,14cyclictest42-21ksoftirqd/308:50:133
1404086994022,14cyclictest42-21ksoftirqd/307:23:003
1404086994019,13cyclictest42-21ksoftirqd/311:45:203
1404086994017,7cyclictest1753-21gnome-shell10:44:553
1404086994015,22cyclictest1469614-21taskset10:10:373
1404086993922,13cyclictest42-21ksoftirqd/308:10:263
1404086993921,6cyclictest42-21ksoftirqd/307:49:453
140408699391,9cyclictest854-21avahi-daemon12:10:183
1404086993917,14cyclictest42-21ksoftirqd/307:34:283
1404086993827,7cyclictest1753-21gnome-shell07:56:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional