You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 18:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sat Mar 28, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1339329279,9sleep20-21swapper/207:06:232
1343729077,9sleep00-21swapper/007:06:570
1341228666,16sleep10-21swapper/107:06:381
1217128263,17sleep30-21swapper/307:05:163
1377299240,23cyclictest0-21swapper/210:44:532
1377299240,19cyclictest0-21swapper/209:10:192
1377399231,22cyclictest0-21swapper/309:22:103
1377399230,19cyclictest0-21swapper/310:55:043
1377299230,23cyclictest0-21swapper/212:16:152
13768992318,4cyclictest25-21ksoftirqd/109:51:281
13768992318,4cyclictest25-21ksoftirqd/109:51:281
1377299220,21cyclictest0-21swapper/209:43:102
1377299220,18cyclictest0-21swapper/211:36:062
1376899221,20cyclictest6533-21diskmemload10:35:261
13773992119,1cyclictest391ksoftirqd/308:59:593
1377299210,20cyclictest0-21swapper/212:31:372
1376899212,18cyclictest6533-21diskmemload11:52:411
1376899210,19cyclictest0-21swapper/109:36:261
1377299200,19cyclictest0-21swapper/210:19:032
308542191,1sleep2141rcu_preempt11:00:012
1377399190,1cyclictest0-21swapper/310:47:533
1377299196,4cyclictest225-21systemd-journal10:24:382
1377299191,17cyclictest0-21swapper/209:24:072
13772991911,4cyclictest225-21systemd-journal09:55:422
1377299190,19cyclictest0-21swapper/211:30:282
13768991917,1cyclictest25-21ksoftirqd/110:41:061
1376899191,3cyclictest0-21swapper/111:21:521
13766991915,1cyclictest475-21in:imuxsock10:24:370
13773991818,0cyclictest0-21swapper/310:18:033
1377399180,2cyclictest0-21swapper/312:37:123
1377399180,17cyclictest0-21swapper/308:26:173
1377299181,16cyclictest8517-21ssh11:25:012
1376899180,17cyclictest0-21swapper/111:09:491
1376699182,2cyclictest0-21swapper/011:39:250
1377399170,17cyclictest0-21swapper/309:57:153
1377299170,17cyclictest0-21swapper/212:36:442
1377299170,17cyclictest0-21swapper/210:00:202
13768991711,6cyclictest25-21ksoftirqd/111:56:421
13766991716,1cyclictest0-21swapper/011:10:070
1377399163,8cyclictest21517-21sh12:30:133
13773991614,1cyclictest0-21swapper/311:43:003
1377399160,16cyclictest0-21swapper/310:09:233
1377299162,12cyclictest0-21swapper/211:45:102
13772991611,2cyclictest27992-21cron12:14:592
1377299160,15cyclictest0-21swapper/211:56:052
1376899163,6cyclictest225-21systemd-journal09:20:171
13768991612,3cyclictest32609-21ssh11:00:201
1376899160,16cyclictest0-21swapper/109:56:161
1376699168,5cyclictest0-21swapper/009:41:160
1376699164,6cyclictest21056-21latency_hist12:30:010
13766991615,1cyclictest0-21swapper/012:03:040
1376699160,1cyclictest0-21swapper/012:18:220
1376699160,15cyclictest0-21swapper/009:30:330
1377399155,9cyclictest1517-21sensors09:25:193
1377399154,10cyclictest0-21swapper/311:21:573
13773991515,0cyclictest0-21swapper/309:41:083
13773991514,1cyclictest39-21ksoftirqd/311:59:233
1377299158,6cyclictest7364-21apache210:26:152
1377299150,15cyclictest0-21swapper/212:28:002
1377299150,15cyclictest0-21swapper/212:21:282
1377299150,14cyclictest0-21swapper/210:08:252
1376899152,12cyclictest20412-21ssh11:31:211
13768991513,1cyclictest25-21ksoftirqd/110:23:001
1376899150,2cyclictest55750irq/128-enp2s0-11:10:141
1376899150,2cyclictest0-21swapper/109:40:161
1376899150,0cyclictest141rcu_preempt10:50:391
13766991514,1cyclictest0-21swapper/010:55:130
13773991411,2cyclictest7741-21grep11:05:153
13773991411,2cyclictest29270-21sort12:15:183
13773991410,3cyclictest28983-21sed08:45:253
1377299149,2cyclictest27048-21sh08:45:002
1377299142,12cyclictest0-21swapper/210:31:162
13772991412,1cyclictest6533-21diskmemload12:01:352
13772991411,2cyclictest3059-21grep11:40:212
13772991411,2cyclictest19643-21wc11:50:182
1376899141,9cyclictest38150irq/131-snd_hda12:35:141
13768991413,1cyclictest221irq_work/110:55:021
13768991411,2cyclictest27095-21sshd12:14:191
1376899140,13cyclictest0-21swapper/110:13:571
13766991412,1cyclictest8347-21strings09:10:190
13766991411,2cyclictest32113-21uniq11:00:170
1377399139,2cyclictest4982-21ssh12:20:173
1377399135,3cyclictest225-21systemd-journal07:15:003
1377399133,7cyclictest21202-21memory08:30:213
13773991310,2cyclictest6042-21grep09:05:283
13773991310,2cyclictest5458-21grep10:25:163
13773991310,2cyclictest32225-21cat11:20:003
13773991310,2cyclictest25147-21cron11:35:003
13773991310,2cyclictest22936-21ssh10:35:203
13773991310,2cyclictest14776-21grep12:25:253
13773991310,2cyclictest11006-21ls12:05:003
1377399130,13cyclictest0-21swapper/311:51:213
13772991312,1cyclictest0-21swapper/210:46:272
13772991311,1cyclictest7145-21ssh09:29:252
13772991311,1cyclictest6533-21diskmemload11:11:182
13772991310,2cyclictest9499-21wc08:05:222
13772991310,2cyclictest15851-21grep07:10:262
1377299130,3cyclictest0-21swapper/211:24:522
1377299130,13cyclictest0-21swapper/210:59:542
1377299130,13cyclictest0-21swapper/209:49:352
1377299130,13cyclictest0-21swapper/209:16:122
1376899139,3cyclictest9009-21grep09:10:261
1376899139,3cyclictest3684-21sed09:45:281
1376899137,5cyclictest29371-21taskset08:49:261
1376899135,4cyclictest13686-21cstates10:30:151
1376899134,8cyclictest7371-21cups-browsed09:30:221
13768991311,1cyclictest0-21swapper/107:15:211
13768991310,2cyclictest634-21cut09:25:131
13768991310,2cyclictest29441-21grep07:40:161
13768991310,2cyclictest2170-21tr07:50:211
13768991310,2cyclictest18580-21head08:25:141
1376899130,1cyclictest0-21swapper/111:16:461
1376699136,6cyclictest30943-21gltestperf10:40:180
1376699135,7cyclictest27064-21munin-run08:44:590
13766991312,1cyclictest0-21swapper/010:45:530
1377399129,2cyclictest7848-21cut09:10:163
1377399129,2cyclictest3220-21ssh09:45:233
1377399129,2cyclictest26079-21wc07:35:013
1377399129,2cyclictest24653-21sh08:39:593
1377399129,2cyclictest22055-21sed12:10:253
1377399129,2cyclictest10660-21cat09:50:133
1377399129,2cyclictest10660-21cat09:50:133
1377399128,3cyclictest23015-21cat07:25:273
1377399124,4cyclictest225-21systemd-journal07:45:003
1377299129,2cyclictest9370-21grep11:05:272
1377299129,2cyclictest332-21grep08:55:182
1377299129,2cyclictest27307-21wc07:35:182
1377299129,2cyclictest22808-21tr08:35:142
1377299129,2cyclictest22319-21ls08:35:002
13772991210,1cyclictest10720-21grep08:10:162
13772991210,1cyclictest0-21swapper/209:33:162
1377299120,1cyclictest0-21swapper/209:50:272
1377299120,1cyclictest0-21swapper/209:50:272
1377299120,12cyclictest0-21swapper/211:17:422
1377299120,11cyclictest15723-21wc10:50:202
1377299120,0cyclictest0-21swapper/210:12:252
1376899129,2cyclictest9889-21grep08:05:261
1376899129,2cyclictest5029-21grep07:55:271
1376899129,2cyclictest32310-21grep07:45:181
1376899129,2cyclictest31343-21tail08:50:261
1376899129,2cyclictest21810-21tr07:25:151
1376899129,2cyclictest21098-21apt-get07:25:011
1376899129,2cyclictest15660-21tr08:20:151
1376899129,2cyclictest13799-21head08:15:171
1376899129,2cyclictest11365-21sed08:10:191
1376899128,3cyclictest7576-21ssh10:07:111
1376899128,3cyclictest17555-21sed09:15:271
1376899127,4cyclictest5914-21ssh12:01:091
1376899123,4cyclictest225-21systemd-journal07:10:161
1376899122,7cyclictest6899-21runrttasks10:25:311
1376899120,12cyclictest0-21swapper/112:15:051
1376699129,2cyclictest28347-21cat08:45:190
1376699129,2cyclictest23371-21grep12:30:260
1376699129,2cyclictest20715-21df10:15:140
1376699129,2cyclictest18721-21ssh09:55:120
1376699129,2cyclictest14113-21grep10:30:170
1376699129,2cyclictest13848-21ssh11:08:530
1376699129,2cyclictest12162-21wc08:10:280
1376699125,6cyclictest24664-21munin-run08:39:590
1376699125,5cyclictest0-21swapper/011:40:290
1376699122,2cyclictest0-21swapper/011:25:130
13766991210,1cyclictest28436-21strings10:00:200
1376699120,11cyclictest157132sleep012:06:580
1377399119,1cyclictest12107-21ls12:05:143
1377399118,2cyclictest9983-21ssh11:45:063
1377399118,2cyclictest8696-21sh09:30:143
1377399118,2cyclictest699-21chrt07:49:243
1377399118,2cyclictest4013-21taskset09:03:223
1377399118,2cyclictest3280-21ssh11:02:153
1377399118,2cyclictest29438-21grep10:20:183
1377399118,2cyclictest28461-21ssh10:00:203
1377399118,2cyclictest27983-21ssh11:35:413
1377399118,2cyclictest26587-21grep08:40:263
1377399118,2cyclictest19074-21ssh09:35:243
1377399118,2cyclictest17533-21rm10:51:123
1377399118,2cyclictest15588-21ssh09:15:143
1377399118,2cyclictest15546-21ssh11:10:003
1377399118,2cyclictest14999-21chrt08:18:163
1377399118,2cyclictest13244-21apt-get10:30:143
1377399118,2cyclictest11564-21ssh10:10:063
1377399118,2cyclictest11378-21ssh11:25:523
1377399114,4cyclictest30166-21ssh10:40:143
1377299119,1cyclictest16843-21grep07:15:142
1377299118,2cyclictest599-21chrt07:48:072
1377299118,2cyclictest5941-21cut08:00:152
1377299118,2cyclictest5559-21sendmail-msp08:00:002
1377299118,2cyclictest30394-21sed07:40:252
1377299118,2cyclictest30003-21ls08:50:152
1377299118,2cyclictest22063-21expr10:35:152
1377299118,2cyclictest19669-21cut07:20:182
1377299118,2cyclictest17009-21sh09:35:082
1377299118,2cyclictest16073-21grep08:20:162
1377299118,2cyclictest12046-21ssh12:05:142
1377299115,3cyclictest225-21systemd-journal08:47:162
1376899119,1cyclictest7012-21sed10:45:161
1376899118,2cyclictest6045-21cut08:00:161
1376899118,2cyclictest31411-21ssh10:02:071
1376899118,2cyclictest30031-21sh11:37:171
1376899118,2cyclictest2804-21ssh11:40:181
1376899118,2cyclictest27982-21sed07:35:261
1376899118,2cyclictest24709-21sed07:30:181
1376899118,2cyclictest23973-21cat08:35:241
1376899118,2cyclictest22742-21ssh10:15:541
1376899118,2cyclictest21404-21ssh12:30:101
1376899118,2cyclictest12947-21ssh11:27:051
1376899118,2cyclictest12017-21ssh12:24:371
1376899118,2cyclictest10655-21expr11:45:151
13768991110,1cyclictest25-21ksoftirqd/112:05:381
1376899110,10cyclictest18986-21ssh12:28:281
1376699119,1cyclictest0-21swapper/011:20:590
1376699118,3cyclictest0-21swapper/009:47:440
1376699118,2cyclictest4642-21expr09:05:140
1376699118,2cyclictest29687-21cat07:40:180
1376699118,2cyclictest25010-21ssh12:12:410
1376699118,2cyclictest18295-21grep07:15:260
1376699118,2cyclictest13785-21sshd08:15:170
1376699118,2cyclictest1284-21sed08:55:280
1376699118,2cyclictest11694-21ssh10:10:120
1376699118,2cyclictest11477-21cut09:50:180
1376699118,2cyclictest11477-21cut09:50:180
1376699111,6cyclictest0-21swapper/010:50:130
13766991110,1cyclictest0-21swapper/010:36:530
1377399108,1cyclictest3850-21ls07:55:153
1377399108,1cyclictest26746-21expr07:35:143
1377399108,1cyclictest20161-21open_files07:20:213
1377399107,2cyclictest7702-21chrt08:01:413
1377399107,2cyclictest30598-21cat08:50:203
1377399107,2cyclictest17338-21head07:15:163
1377399107,2cyclictest1326-21cstates07:50:153
1377399107,2cyclictest11601-21wc08:10:223
1377399102,3cyclictest0-21swapper/308:20:143
1377299108,1cyclictest5085-21cut09:05:172
1377299108,1cyclictest25075-21ntpq07:30:222
1377299108,1cyclictest21940-21ls07:25:152
1377299108,1cyclictest2047-21dpkg09:00:132
1377299107,2cyclictest3156-21perf07:55:002
1377299107,2cyclictest19313-21ls08:25:232
1377299107,2cyclictest14174-21ntpq08:15:202
1376899108,1cyclictest5443-21wc09:05:191
1376899108,1cyclictest32042-21cat08:55:121
1376899108,1cyclictest2317-21tr09:00:151
1376899108,1cyclictest21013-21cat08:30:191
1376899100,6cyclictest26751-21sshd08:41:171
1376699108,2cyclictest4422-21sed10:05:180
1376699108,2cyclictest1083-21rm11:58:580
1376699108,1cyclictest648-21seq07:48:440
1376699108,1cyclictest4931-21ssh10:25:140
1376699108,1cyclictest4645-21cut12:20:150
1376699108,1cyclictest3795-21expr07:55:150
1376699108,1cyclictest32237-21ssh12:35:410
1376699108,1cyclictest29893-21cat08:50:150
1376699108,1cyclictest2849-21ssh09:26:050
1376699108,1cyclictest24192-21ssh11:15:110
1376699108,1cyclictest22215-21sshd08:34:160
1376699108,1cyclictest19172-21ssh09:16:410
1376699108,1cyclictest17097-21ssh11:30:080
1376699108,1cyclictest16922-21sh09:35:050
1376699108,1cyclictest16521-21cat07:15:010
1376699108,1cyclictest15752-21expr08:20:150
1376699108,1cyclictest14298-21ssh11:47:060
1376699107,2cyclictest24342-21expr07:30:160
1376699107,2cyclictest19113-21basename07:20:140
1376699107,2cyclictest18654-21tr11:50:130
1376699104,3cyclictest24859-21head09:20:180
1376699103,6cyclictest1-21systemd08:03:560
1376699103,3cyclictest18089-21grep08:25:110
1376699100,6cyclictest0-21swapper/007:25:140
137739996,2cyclictest9267-21cat08:05:203
137669997,2cyclictest9096-21cat08:05:180
137669997,1cyclictest38652sleep009:01:280
137669997,1cyclictest26707-21expr07:35:150
137669985,2cyclictest1013-21cat07:50:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional