You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot7s.osadl.org (updated Fri May 23, 2025 00:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
795444212865,52sleep20-21swapper/219:06:432
793942211862,24sleep00-21swapper/019:05:000
795500210775,24sleep10-21swapper/119:07:261
795638210664,34sleep30-21swapper/319:09:123
795840997015,42cyclictest0-21swapper/321:55:303
795832992817,10cyclictest0-21swapper/023:32:420
79583299260,25cyclictest0-21swapper/023:46:580
79583299260,25cyclictest0-21swapper/020:02:380
79584099250,25cyclictest0-21swapper/322:08:073
79583999240,24cyclictest0-21swapper/200:16:582
795834992414,4cyclictest803576-21sensors_fan19:25:231
79583999231,14cyclictest844229-21/usr/sbin/munin21:10:102
795832992211,10cyclictest0-21swapper/023:18:330
79584099201,1cyclictest191rcu_preempt00:08:253
79584099198,5cyclictest0-21swapper/323:54:163
79584099192,7cyclictest191rcu_preempt21:19:383
795840991912,7cyclictest0-21swapper/320:35:063
79583999199,4cyclictest0-21swapper/221:30:382
79583999196,1cyclictest0-21swapper/221:49:002
795832991915,4cyclictest0-21swapper/021:58:240
79584099188,10cyclictest0-21swapper/300:18:073
79584099182,0cyclictest0-21swapper/319:43:253
795839991816,1cyclictest0-21swapper/220:04:322
795839991815,2cyclictest1051-21snmpd21:37:172
795834991816,1cyclictest0-21swapper/120:31:181
795834991815,2cyclictest1051-21snmpd00:03:221
795834991812,5cyclictest0-21swapper/121:00:051
79583499180,0cyclictest0-21swapper/122:16:131
79583299187,10cyclictest0-21swapper/022:22:070
795839991714,2cyclictest1051-21snmpd22:39:172
79583999170,1cyclictest0-21swapper/220:35:162
79583999170,17cyclictest0-21swapper/223:29:122
79583999170,17cyclictest0-21swapper/220:15:512
79583499170,17cyclictest0-21swapper/122:54:291
79583499170,16cyclictest0-21swapper/123:50:121
79583499170,16cyclictest0-21swapper/123:44:041
79583499170,16cyclictest0-21swapper/122:46:281
79583499170,16cyclictest0-21swapper/122:37:471
79583499170,16cyclictest0-21swapper/119:46:071
79583299170,17cyclictest0-21swapper/022:28:520
79583299170,17cyclictest0-21swapper/021:47:360
79583299170,17cyclictest0-21swapper/020:46:280
79583299170,17cyclictest0-21swapper/020:13:500
79583299170,17cyclictest0-21swapper/019:59:150
79583299170,15cyclictest0-21swapper/021:10:160
79584099166,1cyclictest0-21swapper/300:03:243
79584099162,0cyclictest0-21swapper/300:22:173
795840991610,6cyclictest0-21swapper/322:50:233
79584099160,7cyclictest0-21swapper/321:45:093
79584099160,16cyclictest0-21swapper/323:05:353
79584099160,16cyclictest0-21swapper/320:32:493
79584099160,16cyclictest0-21swapper/300:29:073
79583999160,16cyclictest0-21swapper/223:37:142
79583999160,16cyclictest0-21swapper/200:25:262
79583499165,10cyclictest0-21swapper/122:25:581
795834991613,2cyclictest1051-21snmpd23:57:371
795834991613,2cyclictest1051-21snmpd21:45:331
79583499160,16cyclictest0-21swapper/123:14:141
79583499160,16cyclictest0-21swapper/120:28:421
79583299166,9cyclictest0-21swapper/021:08:220
79583299161,6cyclictest0-21swapper/022:48:340
795832991615,1cyclictest0-21swapper/000:23:250
795832991614,1cyclictest0-21swapper/022:10:100
795832991611,5cyclictest0-21swapper/000:33:250
79583299160,7cyclictest0-21swapper/000:14:330
79583299160,16cyclictest0-21swapper/022:54:380
79584099157,0cyclictest0-21swapper/323:00:143
795840991514,1cyclictest0-21swapper/321:20:173
795840991511,4cyclictest0-21swapper/319:12:013
79584099150,6cyclictest0-21swapper/320:29:453
79584099150,1cyclictest0-21swapper/322:29:573
79584099150,1cyclictest0-21swapper/320:05:183
79584099150,15cyclictest0-21swapper/321:28:563
79584099150,10cyclictest0-21swapper/322:44:563
79583999155,9cyclictest0-21swapper/221:40:092
79583999155,9cyclictest0-21swapper/221:40:082
795839991513,1cyclictest1051-21snmpd22:58:582
795839991511,4cyclictest0-21swapper/222:47:202
795839991510,5cyclictest0-21swapper/223:30:222
79583999150,7cyclictest0-21swapper/220:34:152
79583999150,15cyclictest0-21swapper/223:40:172
79583999150,15cyclictest0-21swapper/220:49:292
79583499155,4cyclictest0-21swapper/119:16:301
79583499152,13cyclictest0-21swapper/121:06:011
795834991511,4cyclictest0-21swapper/119:14:071
79583499150,6cyclictest0-21swapper/100:20:521
79583499150,15cyclictest0-21swapper/120:02:351
79583499150,15cyclictest0-21swapper/119:33:371
79583499150,14cyclictest0-21swapper/119:40:161
795832991512,3cyclictest0-21swapper/022:30:180
79583299150,6cyclictest0-21swapper/022:06:120
79583299150,15cyclictest0-21swapper/020:43:220
79584099144,10cyclictest0-21swapper/322:31:163
795840991411,2cyclictest1051-21snmpd23:34:453
795840991411,2cyclictest1051-21snmpd22:18:543
795840991411,2cyclictest1051-21snmpd21:05:343
795840991410,4cyclictest0-21swapper/323:55:343
79584099140,1cyclictest0-21swapper/322:45:203
79584099140,1cyclictest0-21swapper/319:50:393
79584099140,12cyclictest50-21ksoftirqd/322:03:153
795839991413,1cyclictest0-21swapper/200:34:572
795839991410,4cyclictest0-21swapper/223:16:032
795839991410,4cyclictest0-21swapper/223:00:352
795839991410,4cyclictest0-21swapper/221:02:492
79583499149,1cyclictest0-21swapper/122:42:411
795834991411,2cyclictest1051-21snmpd20:10:081
795834991410,4cyclictest0-21swapper/123:49:581
79583499140,5cyclictest0-21swapper/123:02:531
79583499140,13cyclictest0-21swapper/120:05:161
79583499140,13cyclictest0-21swapper/119:20:151
795832991412,1cyclictest1051-21snmpd23:58:190
795832991410,4cyclictest0-21swapper/021:53:000
79584099139,4cyclictest0-21swapper/323:11:363
79584099136,7cyclictest0-21swapper/322:39:353
795840991312,1cyclictest0-21swapper/321:00:183
79584099130,13cyclictest0-21swapper/319:20:153
79583999139,4cyclictest0-21swapper/220:56:312
79583999138,5cyclictest0-21swapper/219:52:542
79583999132,1cyclictest0-21swapper/219:28:172
795839991312,1cyclictest0-21swapper/222:50:162
795839991312,1cyclictest0-21swapper/222:10:142
795839991312,1cyclictest0-21swapper/222:05:122
795839991312,1cyclictest0-21swapper/220:10:152
795839991310,2cyclictest870461-21fschecks_time22:15:122
79583999130,1cyclictest0-21swapper/223:10:132
79583999130,1cyclictest0-21swapper/219:40:192
79583999130,13cyclictest0-21swapper/222:25:142
79583999130,13cyclictest0-21swapper/219:20:162
79583999130,13cyclictest0-21swapper/219:10:182
795834991312,1cyclictest0-21swapper/100:35:141
795834991310,2cyclictest846698-21fschecks_time21:15:131
795834991310,2cyclictest1051-21snmpd22:31:301
795834991310,2cyclictest1051-21snmpd20:35:271
79583499130,13cyclictest0-21swapper/122:01:021
79583499130,12cyclictest0-21swapper/121:50:171
79583299138,1cyclictest0-21swapper/020:51:250
79583299133,9cyclictest0-21swapper/019:50:320
795832991312,1cyclictest0-21swapper/023:50:200
795832991312,1cyclictest0-21swapper/019:45:140
795832991310,2cyclictest1051-21snmpd20:25:510
79583299130,1cyclictest0-21swapper/019:30:130
13935799130,2rtkit-daemon139356-21rtkit-daemon23:27:100
13935799130,2rtkit-daemon139356-21rtkit-daemon23:12:000
13935799130,2rtkit-daemon139356-21rtkit-daemon19:43:360
79584099129,2cyclictest860466-21fschecks_count21:50:153
79584099129,2cyclictest1051-21snmpd20:58:223
79584099128,1cyclictest0-21swapper/323:39:363
79584099128,1cyclictest0-21swapper/320:41:483
795840991211,1cyclictest894107-21df_abs23:15:163
79584099120,1cyclictest0-21swapper/321:40:543
79584099120,1cyclictest0-21swapper/321:40:543
79583999129,2cyclictest1051-21snmpd23:21:082
79583999129,2cyclictest1051-21snmpd22:20:432
79583999129,2cyclictest1051-21snmpd20:44:492
79583999129,2cyclictest1051-21snmpd20:26:392
79583999129,2cyclictest1051-21snmpd19:33:282
79583999129,2cyclictest1051-21snmpd00:08:012
79583999123,9cyclictest0-21swapper/222:00:122
79583999123,9cyclictest0-21swapper/219:57:442
79583999122,9cyclictest925953-21df_inode00:35:142
79583499129,2cyclictest873777-21cut22:25:001
79583499129,2cyclictest1051-21snmpd23:30:301
79583499129,2cyclictest1051-21snmpd23:27:141
79583499129,2cyclictest1051-21snmpd23:19:151
79583499129,2cyclictest1051-21snmpd23:08:511
79583499129,2cyclictest1051-21snmpd21:41:121
79583499129,2cyclictest1051-21snmpd21:41:121
79583499129,2cyclictest1051-21snmpd19:59:531
79583499129,2cyclictest1051-21snmpd00:27:051
79583499129,2cyclictest1045-21nmbd22:55:331
79583499129,2cyclictest1045-21nmbd19:35:221
79583499128,4cyclictest0-21swapper/121:37:561
79583499122,4cyclictest0-21swapper/122:12:311
795834991211,1cyclictest0-21swapper/120:45:181
795834991210,1cyclictest1051-21snmpd21:11:191
79583299129,2cyclictest1051-21snmpd20:33:450
79583299128,4cyclictest0-21swapper/022:15:550
79583299128,4cyclictest0-21swapper/022:00:210
79583299122,10cyclictest0-21swapper/021:27:050
795832991211,1cyclictest0-21swapper/020:20:200
79584099119,2cyclictest836565-21df_abs20:50:133
79584099119,1cyclictest844806-21sort21:10:133
79584099119,1cyclictest1051-21snmpd23:25:003
79584099118,2cyclictest152740-21JS16:39:433
79584099117,4cyclictest0-21swapper/321:36:473
79584099117,1cyclictest0-21swapper/322:59:103
79584099117,1cyclictest0-21swapper/320:24:243
795840991111,0cyclictest0-21swapper/300:30:193
795840991110,1cyclictest907200-21taskset23:46:083
795840991110,1cyclictest1051-21snmpd19:56:443
79584099110,10cyclictest905143-21users23:40:223
79583999119,1cyclictest1051-21snmpd00:10:382
79583999118,2cyclictest836382-21cpuspeed_turbos20:50:122
79583999118,2cyclictest803-21in:imjournal19:14:592
79583999118,2cyclictest1051-21snmpd19:39:222
79583999117,4cyclictest0-21swapper/221:52:212
79583999116,5cyclictest0-21swapper/222:32:212
79583999111,10cyclictest0-21swapper/223:55:432
79583999111,10cyclictest0-21swapper/223:54:032
795839991110,1cyclictest0-21swapper/220:20:182
79583999110,4cyclictest818523-21grep20:05:122
79583999110,11cyclictest0-21swapper/221:55:142
79583499118,2cyclictest798-21systemd-machine00:13:481
79583499118,2cyclictest1051-21snmpd21:56:331
79583499118,2cyclictest1051-21snmpd21:33:231
79583499118,2cyclictest1051-21snmpd20:42:091
79583499118,2cyclictest1051-21snmpd20:19:181
79583499118,2cyclictest1045-21nmbd23:23:531
79583499117,4cyclictest0-21swapper/120:50:371
795834991110,1cyclictest0-21swapper/120:23:311
79583499110,10cyclictest0-21swapper/100:30:011
79583299119,2cyclictest1051-21snmpd19:20:240
79583299119,1cyclictest1051-21snmpd22:38:050
79583299119,1cyclictest1051-21snmpd21:15:030
79583299118,2cyclictest1051-21snmpd22:43:200
79583299118,2cyclictest1051-21snmpd21:00:130
79583299117,3cyclictest857886-21sh21:44:590
79583299117,3cyclictest857886-21sh21:44:590
79583299117,1cyclictest0-21swapper/000:38:010
79583299116,5cyclictest0-21swapper/019:14:300
79583299112,9cyclictest0-21swapper/021:22:320
79583299112,9cyclictest0-21swapper/000:09:550
795832991110,1cyclictest1051-21snmpd21:30:410
79583299110,10cyclictest0-21swapper/020:19:540
79583299110,10cyclictest0-21swapper/000:25:050
79584099109,1cyclictest806815-21sed19:35:123
79584099109,1cyclictest1051-21snmpd19:32:583
79584099109,1cyclictest1051-21snmpd19:25:073
79584099108,2cyclictest810850-21fschecks_count19:45:133
79584099108,2cyclictest140665-21gmain23:20:193
79584099108,1cyclictest1051-21snmpd00:36:113
79584099107,2cyclictest1051-21snmpd20:49:493
79584099103,7cyclictest0-21swapper/322:20:133
79584099100,10cyclictest0-21swapper/320:16:093
79584099100,10cyclictest0-21swapper/320:10:053
79584099100,10cyclictest0-21swapper/319:15:143
79583999109,1cyclictest1051-21snmpd19:47:102
79583999106,4cyclictest0-21swapper/200:22:352
79583999100,9cyclictest842669-21fschecks_time21:05:152
79583999100,10cyclictest0-21swapper/223:05:162
79583999100,10cyclictest0-21swapper/222:42:502
79583999100,10cyclictest0-21swapper/221:25:232
79583999100,10cyclictest0-21swapper/221:20:212
79583499107,2cyclictest838644-21fschecks_count20:55:171
79583499107,2cyclictest1051-21snmpd21:21:201
79583499107,2cyclictest1051-21snmpd19:52:591
79583499106,1cyclictest0-21swapper/123:37:081
79583499100,10cyclictest0-21swapper/122:08:251
79583499100,10cyclictest0-21swapper/121:27:201
79583299108,1cyclictest1051-21snmpd23:44:370
79583299108,1cyclictest1051-21snmpd21:35:170
79583299108,1cyclictest1051-21snmpd19:36:010
79583299106,4cyclictest0-21swapper/022:56:020
79583299101,9cyclictest0-21swapper/000:18:320
79583299101,3cyclictest818898-21grep20:05:130
79583299100,10cyclictest0-21swapper/023:20:450
79583299100,10cyclictest0-21swapper/020:58:400
79583299100,10cyclictest0-21swapper/019:19:520
13935799100,7rtkit-daemon153283-21gmain20:39:180
7958409998,1cyclictest853045-21ntp_states21:30:183
7958409998,1cyclictest1058-21runrttasks19:59:593
7958409996,2cyclictest152740-21JS16:39:433
7958399990,8cyclictest911717-21/usr/sbin/munin00:00:112
7958349996,2cyclictest1051-21snmpd00:16:041
7958349996,2cyclictest1051-21snmpd00:06:481
7958329997,2cyclictest1051-21snmpd23:09:000
7958329997,1cyclictest1051-21snmpd19:26:500
7958329996,2cyclictest912062-21fschecks_count00:00:140
7958399986,2cyclictest1051-21snmpd23:47:102
7958399980,7cyclictest846212-21/usr/sbin/munin21:15:132
7958329987,1cyclictest1051-21snmpd23:04:560
7958329980,1cyclictest0-21swapper/023:35:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional