You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-09 - 11:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Wed Sep 09, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2709529783,9sleep00-21swapper/019:06:370
2706529380,9sleep20-21swapper/219:06:152
2694628769,13sleep10-21swapper/119:05:291
2734428371,8sleep30-21swapper/319:09:543
2744699252,23cyclictest0-21swapper/223:06:492
2744699250,25cyclictest0-21swapper/200:06:422
2744299241,22cyclictest2375-21taskset22:15:261
2744299232,2cyclictest0-21swapper/123:40:171
27448992220,1cyclictest0-21swapper/323:04:413
27442992121,0cyclictest0-21swapper/123:36:511
2744699200,20cyclictest0-21swapper/222:38:202
2744699200,20cyclictest0-21swapper/222:08:512
2744699200,20cyclictest0-21swapper/222:08:512
2744699200,20cyclictest0-21swapper/200:03:512
2744699200,19cyclictest0-21swapper/221:49:082
2744299200,19cyclictest0-21swapper/122:01:511
2744099200,1cyclictest0-21swapper/000:10:240
2744099200,19cyclictest0-21swapper/023:35:150
27448991917,2cyclictest23614-21kworker/3:123:08:233
2744899190,18cyclictest23614-21kworker/3:122:50:233
2744899190,18cyclictest23614-21kworker/3:122:22:543
2744699191,1cyclictest20226-21diskmemload23:31:362
2744699190,19cyclictest0-21swapper/223:35:142
2744699190,19cyclictest0-21swapper/223:15:412
2744699190,18cyclictest0-21swapper/221:20:082
2744099197,10cyclictest0-21swapper/020:50:130
317082180,2chrt31239-21apt-get21:55:020
27448991817,1cyclictest23614-21kworker/3:122:39:233
27448991817,1cyclictest23614-21kworker/3:121:57:233
27448991817,1cyclictest23614-21kworker/3:121:39:343
27448991817,1cyclictest23614-21kworker/3:121:24:133
2744899180,18cyclictest23614-21kworker/3:123:46:543
2744899180,18cyclictest23614-21kworker/3:121:42:033
2744899180,17cyclictest23614-21kworker/3:121:52:133
2744899180,17cyclictest0-21swapper/321:12:593
2744699180,17cyclictest13281-21grep23:20:152
2744299180,1cyclictest0-21swapper/123:10:231
2744299180,1cyclictest0-21swapper/122:39:101
128152180,0sleep20-21swapper/200:36:052
27448991716,1cyclictest23614-21kworker/3:123:55:033
27448991716,1cyclictest23614-21kworker/3:123:50:333
27448991716,1cyclictest23614-21kworker/3:123:36:233
27448991716,1cyclictest23614-21kworker/3:123:25:533
27448991716,1cyclictest23614-21kworker/3:123:23:143
27448991716,1cyclictest23614-21kworker/3:122:57:033
27448991716,1cyclictest23614-21kworker/3:122:47:533
27448991716,1cyclictest23614-21kworker/3:122:15:333
27448991716,1cyclictest23614-21kworker/3:122:04:043
27448991716,1cyclictest23614-21kworker/3:121:32:533
27448991716,1cyclictest23614-21kworker/3:121:18:233
2744899170,17cyclictest23614-21kworker/3:123:43:533
2744899170,17cyclictest23614-21kworker/3:122:13:233
2744899170,17cyclictest23614-21kworker/3:122:08:033
2744899170,17cyclictest23614-21kworker/3:122:08:033
2744899170,17cyclictest23614-21kworker/3:100:03:333
27446991715,1cyclictest0-21swapper/221:19:372
2744699170,16cyclictest0-21swapper/223:45:082
2744699170,0cyclictest0-21swapper/223:11:572
2744299171,15cyclictest0-21swapper/122:25:191
2744099174,7cyclictest30669-21basename23:50:000
27448991615,1cyclictest29352-21kworker/3:000:28:133
27448991615,1cyclictest23614-21kworker/3:121:25:533
2744699160,1cyclictest0-21swapper/222:47:342
2744699160,16cyclictest0-21swapper/222:30:322
2744699160,15cyclictest0-21swapper/222:19:542
2744699160,15cyclictest0-21swapper/221:31:232
2744299160,2cyclictest22375-21fschecks_time20:05:171
2744099160,9cyclictest12626-21grep23:20:000
2744099160,16cyclictest0-21swapper/023:21:000
2744099160,13cyclictest0-21swapper/021:30:170
27448991514,1cyclictest29352-21kworker/3:000:11:233
27448991514,1cyclictest23614-21kworker/3:123:12:133
2744899150,15cyclictest0-21swapper/322:44:383
27446991512,2cyclictest4506-21idleruntime-cro19:30:002
27446991511,3cyclictest24099-21cut21:50:192
2744699150,1cyclictest0-21swapper/221:44:022
2744699150,15cyclictest0-21swapper/222:53:172
2744699150,15cyclictest0-21swapper/221:25:512
2744699150,15cyclictest0-21swapper/200:32:402
2744699150,15cyclictest0-21swapper/200:27:452
2744699150,15cyclictest0-21swapper/200:14:062
2744299152,12cyclictest38150irq/131-snd_hda19:30:141
2744299150,1cyclictest0-21swapper/100:10:181
2744099150,1cyclictest0-21swapper/000:26:190
27448991410,3cyclictest6895-21awk19:35:003
27448991410,3cyclictest29325-21systemd00:07:563
27448991410,2cyclictest9450-21sed00:15:163
27446991410,3cyclictest32555-21head20:25:212
2744699140,14cyclictest0-21swapper/223:53:012
2744699140,13cyclictest0-21swapper/223:04:012
2744699140,0cyclictest0-21swapper/221:10:032
2744299145,7cyclictest29363-31find00:08:021
27442991413,1cyclictest25-21ksoftirqd/100:15:501
27442991411,2cyclictest15330-21ssh21:45:141
2744299140,1cyclictest0-21swapper/122:49:271
2744099147,6cyclictest22855-21awk21:50:000
27440991413,1cyclictest13-21ksoftirqd/022:03:040
27440991411,2cyclictest8046-21tr23:55:170
27440991411,2cyclictest25583-21basename22:30:000
27440991410,3cyclictest17963-21grep00:20:170
27440991410,3cyclictest13799-21grep19:45:240
27440991410,2cyclictest4207-21grep22:55:170
2744099140,9cyclictest32173-21perf00:10:000
2744099140,1cyclictest0-21swapper/023:43:010
173472140,2chrt17346-21ssh00:00:261
2744899139,3cyclictest16501-21sed21:00:183
27448991312,1cyclictest0-21swapper/321:46:573
27448991310,2cyclictest30721-21cut19:15:183
27448991310,2cyclictest18491-21tail22:25:163
2744899130,1cyclictest0-21swapper/323:31:033
2744699139,3cyclictest9829-21tr20:45:232
2744699139,3cyclictest32173-21munin-plugin-st19:20:002
2744699139,3cyclictest18581-21sed19:55:262
2744699138,3cyclictest24480-21ssh22:10:022
27446991310,2cyclictest9355-21sed22:00:222
27446991310,2cyclictest26941-21tr20:15:152
27446991310,2cyclictest24354-21grep20:10:162
27446991310,2cyclictest24354-21grep20:10:162
2744699130,1cyclictest0-21swapper/222:42:492
2744299139,3cyclictest4896-21tr20:35:201
2744299139,3cyclictest3660-21grep22:55:141
2744299139,2cyclictest30720-21grep21:35:151
27442991312,1cyclictest0-21swapper/123:01:321
2744299131,1cyclictest141rcu_preempt21:30:221
27442991310,2cyclictest6500-21cat23:15:261
27442991310,2cyclictest21430-21cron20:05:001
27442991310,2cyclictest18481-21head00:20:201
27442991310,2cyclictest10730-21tr19:40:211
2744299130,13cyclictest0-21swapper/121:15:281
2744299130,12cyclictest6237-21cut20:40:161
2744299130,12cyclictest25034-21tr22:10:141
2744099139,3cyclictest2844-21cron22:54:590
27440991312,1cyclictest0-21swapper/021:12:230
27440991310,2cyclictest7386-21sed20:40:220
27440991310,2cyclictest29471-21basename21:15:160
27440991310,2cyclictest19255-21cut22:45:130
27440991310,2cyclictest16701-21sh00:20:000
2744099130,12cyclictest18266-21grep22:05:250
2744099130,12cyclictest18266-21grep22:05:250
2744099130,11cyclictest0-21swapper/021:25:340
2744899129,2cyclictest612-21grep19:20:163
2744899128,3cyclictest4195-21expr20:35:153
2744899128,2cyclictest9737-21cut19:40:163
2744899128,2cyclictest12779-21grep19:45:143
2744899127,2cyclictest24127-21ssh00:24:163
2744899123,5cyclictest225-21systemd-journal20:05:003
27448991211,1cyclictest0-21swapper/322:31:033
2744899120,6cyclictest17832-21dump-pmu-power00:40:003
2744899120,11cyclictest3069-21sed20:30:283
2744699129,2cyclictest9671-21grep00:15:172
2744699129,2cyclictest7956-21grep23:55:162
2744699129,2cyclictest6386-21sed19:30:272
2744699129,2cyclictest31001-21expr21:35:162
2744699129,2cyclictest18348-21wc00:20:192
2744699129,2cyclictest10345-21grep19:40:192
2744699128,2cyclictest15680-21tr23:40:212
2744699123,5cyclictest225-21systemd-journal19:15:002
2744699121,4cyclictest0-21swapper/222:57:372
27446991210,1cyclictest16979-21cron22:25:012
27446991210,1cyclictest14358-21cat19:50:002
2744699120,2cyclictest0-21swapper/221:56:262
2744299129,2cyclictest7878-21grep21:40:191
2744299129,2cyclictest3105-21tail19:25:161
2744299129,2cyclictest15819-21wc19:50:221
2744299128,3cyclictest3525-21grep00:30:251
2744299128,3cyclictest32166-21tail20:25:181
2744299128,3cyclictest15054-21head20:55:271
2744299128,3cyclictest10969-21grep22:40:161
2744299123,2cyclictest225-21systemd-journal19:15:001
27442991210,1cyclictest20226-21diskmemload23:58:251
2744299120,12cyclictest0-21swapper/121:11:231
2744299120,11cyclictest0-21swapper/123:09:321
2744099129,2cyclictest32762-21grep23:50:210
2744099129,2cyclictest32328-21wc19:20:000
2744099129,2cyclictest25994-21ssh22:30:100
2744099129,2cyclictest2422-21grep00:30:160
2744099129,2cyclictest23922-21cat20:10:000
2744099129,2cyclictest22005-21tr23:25:150
2744099129,2cyclictest19248-21wc20:00:010
2744099129,2cyclictest16655-21cron19:55:000
2744099128,3cyclictest17867-21awk21:05:000
2744099128,3cyclictest15452-21apt20:59:590
2744099128,3cyclictest1121-21cron20:30:000
2744099125,5cyclictest4778-21runrttasks19:30:130
2744899118,2cyclictest8015-21ssh00:33:523
2744899118,2cyclictest6783-21ssh23:15:313
2744899118,2cyclictest3124-21head19:25:163
2744899118,2cyclictest28389-21tail19:10:183
2744899118,2cyclictest18847-21head21:05:173
2744899118,2cyclictest17648-21head19:55:173
2744899116,2cyclictest225-21systemd-journal19:40:003
2744899112,5cyclictest24087-21apt-get20:10:143
2744899112,5cyclictest24087-21apt-get20:10:143
2744699118,2cyclictest702-21cut19:20:162
2744699118,2cyclictest23621-21sed23:25:262
2744699118,2cyclictest19575-21tr20:00:162
2744699118,2cyclictest17969-21cat21:05:012
2744699118,2cyclictest17967-21grep22:25:142
2744699118,2cyclictest12068-21cat20:50:212
2744699113,4cyclictest225-21systemd-journal19:38:182
2744699110,1cyclictest0-21swapper/220:05:142
2744299119,1cyclictest8879-21tr20:45:161
2744299119,1cyclictest13450-21tr23:20:161
2744299118,2cyclictest908-21ssh21:55:231
2744299118,2cyclictest788-21grep19:20:161
2744299118,2cyclictest2587-21ssh22:54:471
2744299118,2cyclictest23049-21tr23:45:151
2744299118,2cyclictest20294-21cat21:10:001
2744299118,2cyclictest14281-21cron19:50:001
2744299118,2cyclictest14185-21ssh22:22:491
2744299118,2cyclictest12650-21ssh21:24:311
2744299118,2cyclictest12566-21ssh00:35:531
2744299117,3cyclictest8651-21ls19:35:251
2744299117,3cyclictest25388-21grep21:50:291
2744299116,4cyclictest28664-21ssh00:26:161
2744299115,2cyclictest30688-21df23:30:151
2744299114,6cyclictest241rcuc/122:30:141
2744299113,4cyclictest225-21systemd-journal22:07:301
2744299113,4cyclictest225-21systemd-journal22:07:301
2744299111,2cyclictest0-21swapper/123:50:131
2744099119,1cyclictest3524-21awk19:25:190
2744099118,2cyclictest9273-21rm21:21:530
2744099118,2cyclictest8256-21uname19:35:200
2744099118,2cyclictest6250-21ssh21:40:010
2744099118,2cyclictest31461-21ssh23:30:220
2744099118,2cyclictest30807-21ssh23:10:300
2744099118,2cyclictest2841-21ssh22:15:320
2744099118,2cyclictest27937-21grep19:10:160
2744099118,2cyclictest27267-21grep20:15:170
2744099118,2cyclictest26564-21ssh21:51:230
2744099118,2cyclictest25995-21tr22:10:190
2744099118,2cyclictest2047-21ssh22:35:060
2744099118,2cyclictest20454-21tr23:05:140
2744099118,2cyclictest18603-21df21:05:150
2744099118,2cyclictest1322-21ssh21:36:230
2744099118,2cyclictest11425-21ssh22:20:390
2744099118,2cyclictest11254-21grep19:40:270
2744099118,2cyclictest11234-21rm22:40:170
2744099118,2cyclictest10034-21ssh00:35:150
2744099115,2cyclictest225-21systemd-journal20:35:000
2744099113,7cyclictest19462-21ssh00:01:500
2744899108,1cyclictest31447-21apache_processe20:25:143
2744899108,1cyclictest27598-21uname20:15:193
2744899108,1cyclictest15691-21awk19:50:213
2744899108,1cyclictest14251-21interrupts20:55:193
2744899107,2cyclictest6129-21taskset20:40:073
2744899107,2cyclictest21902-21sort20:05:143
2744899106,3cyclictest30186-21cut20:20:193
2744699108,1cyclictest19979-21chrt21:06:592
2744699108,1cyclictest14636-21basename19:50:152
2744699107,2cyclictest6834-21grep20:40:172
2744699107,2cyclictest29339-21tr20:20:142
2744699107,2cyclictest2933-21cat20:30:272
2744299108,1cyclictest31983-21chrt19:17:521
2744299108,1cyclictest29201-21basename20:20:141
2744299108,1cyclictest26650-21cat20:15:131
2744299108,1cyclictest25390-21rt-features20:10:231
2744299108,1cyclictest25390-21rt-features20:10:231
2744299108,1cyclictest2214-21cat20:30:191
2744299108,1cyclictest21631-21ssh23:25:111
2744299108,1cyclictest17181-21cut19:55:161
2744299108,1cyclictest15691-21grep21:25:261
2744299106,2cyclictest11407-21expr20:50:151
2744299104,5cyclictest23977-21munin-node21:04:171
2744099108,2cyclictest30196-21ntpq20:20:190
2744099108,1cyclictest8946-21expr20:45:160
2744099108,1cyclictest807-21smartctl19:20:170
2744099108,1cyclictest55232sleep020:35:320
2744099108,1cyclictest24522-21expr20:10:160
2744099108,1cyclictest24522-21expr20:10:160
2744099108,1cyclictest20704-21basename20:00:230
2744099108,1cyclictest11633-21sh23:00:080
274489997,1cyclictest127792sleep320:51:183
274489997,1cyclictest10505-21chrt20:47:333
274469997,1cyclictest15100-21taskset20:55:392
274469993,5cyclictest5378-21grep20:35:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional