Copy-pasting nodes (#7618)

Closes #6261

- Adds support for copy-pasting nodes with `cmd + C` and `cmd + V` shortcuts.
- Only a single, currently selected node will be copied. Adding support for multiple node copies seems easy, though (but was out of the scope of the task).
- We use a custom data format for clipboard content. Node's metadata is also copied, so opened visualizations are preserved. However, the visualization's size is not preserved, as we do not store this info in metadata.
- For custom format to work, we use a pretty new feature called [Clipboard pickling](https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md), but it is available in Electron and in most browsers already.
- Pasting plain text from other applications (or from Enso, if the code is copied in edit mode) is supported and is currently enabled. There are some security concerns related to this, though. I will create a separate issue/discussion for that.
- Undo/redo works as you expect.
- New node is pasted at the cursor position.


https://github.com/enso-org/enso/assets/6566674/7a04d941-19f7-4a39-9bce-0e554af50ba3
This commit is contained in:
Ilya Bogdanov 2023-08-30 17:04:21 +04:00 committed by GitHub
parent c834847c48
commit b49cc25d38
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 419 additions and 36 deletions

View File

@ -219,6 +219,10 @@
<kbd>enter</kbd>][7527]
- [Connections to lamdas are displayed correctly][7550]. It is possible to drag
a connection to any expression inside the lambda body.
- [Copying and pasting a single node][7618]. Using the common
<kbd>cmd</kbd>+<kbd>C</kbd> and <kbd>cmd</kbd>+<kbd>V</kbd> shortcuts, it is
now possible to copy a single selected node and paste its code to the graph or
another program.
[5910]: https://github.com/enso-org/enso/pull/5910
[6279]: https://github.com/enso-org/enso/pull/6279
@ -245,6 +249,7 @@
[7311]: https://github.com/enso-org/enso/pull/7311
[7527]: https://github.com/enso-org/enso/pull/7527
[7550]: https://github.com/enso-org/enso/pull/7550
[7618]: https://github.com/enso-org/enso/pull/7618
#### EnsoGL (rendering engine)

View File

@ -458,7 +458,7 @@ impl NodeAst {
/// AST of the node's expression. Typically no external user wants to access it directly. Use
/// [`Self::expression`] instead.
fn whole_expression(&self) -> &Ast {
pub fn whole_expression(&self) -> &Ast {
match self {
NodeAst::Binding { infix, .. } => &infix.rarg,
NodeAst::Expression { ast, .. } => ast,

View File

@ -53,6 +53,8 @@ broken and require further investigation.
| <kbd>cmd</kbd>+<kbd>alt</kbd>+<kbd>r</kbd> | Re-execute the program |
| <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>k</kbd> | Switch the execution environment to Design. |
| <kbd>cmd</kbd>+<kbd>shift</kbd>+<kbd>l</kbd> | Switch the execution environment to Live. |
| <kbd>cmd</kbd>+<kbd>c</kbd> | Copy the selected nodes to the clipboard. |
| <kbd>cmd</kbd>+<kbd>v</kbd> | Paste a node from the clipboard at the mouse cursor position. |
#### Navigation

View File

@ -46,6 +46,10 @@ pub use double_representation::graph::LocationHint;
mod clipboard;
// ==============
// === Errors ===
// ==============
@ -930,6 +934,22 @@ impl Handle {
Ok(())
}
/// Copy the node to clipboard. See `clipboard` module documentation for details.
pub fn copy_node(&self, id: ast::Id) -> FallibleResult {
let graph = GraphInfo::from_definition(self.definition()?.item);
let node = graph.locate_node(id)?;
let expression = node.whole_expression().repr();
let metadata = self.module.node_metadata(id).ok();
clipboard::copy_node(expression, metadata)?;
Ok(())
}
/// Paste a node from clipboard at cursor position. See `clipboard` module documentation for
/// details.
pub fn paste_node(&self, cursor_pos: Vector2, on_error: fn(String)) {
clipboard::paste_node(self, cursor_pos, on_error);
}
/// Sets the given's node expression.
#[profile(Debug)]
pub fn set_expression(&self, id: ast::Id, expression_text: impl Str) -> FallibleResult {

View File

@ -0,0 +1,172 @@
//! Copy-pasting nodes using the clipboard.
//!
//! # Clipboard Content Format
//!
//! We use a JSON-encoded [`ClipboardContent`] structure, marked with our custom [`MIME_TYPE`].
//! This way, we have a separate clipboard format for our application and can extend it in the
//! future.
//! We also support plain text pasting to make it easier to paste the content from other
//! applications, but only if the [`PLAIN_TEXT_PASTING_ENABLED`] is `true`. Allowing pasting plain
//! text can bring unnecessary security risks, like the execution of malicious code immediately
//! after pasting.
//!
//! To copy the node as plain text, the user can enter the editing node, select the node expression,
//! and copy it to the clipboard using the [`ensogl::Text`] functionality.
use crate::prelude::*;
use crate::controller::graph::Handle;
use crate::controller::graph::NewNodeInfo;
use crate::model::module::NodeMetadata;
use ensogl::system::web::clipboard;
use serde::Deserialize;
use serde::Serialize;
// =================
// === Constants ===
// =================
/// We use the `web` prefix to be able to use a custom MIME type. Typically browsers support a
/// restricted set of MIME types in the clipboard.
/// See [Clipboard pickling](https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md).
///
/// `application/enso` is not an officially registered MIME-type (yet), but it is not important for
/// our purposes.
const MIME_TYPE: &str = "web application/enso";
/// Whether to allow pasting nodes from plain text.
const PLAIN_TEXT_PASTING_ENABLED: bool = true;
// ==============
// === Errors ===
// ==============
#[derive(Debug, Clone, PartialEq, failure::Fail)]
#[fail(
display = "`application/enso` MIME-type is used, but clipboard content has incorrect format."
)]
pub struct InvalidFormatError;
/// Clipboard payload.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
enum ClipboardContent {
/// A single node that was copied from the application.
Node(CopiedNode),
}
/// A single node that was copied from the application.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
struct CopiedNode {
/// A whole node's expression (without a pattern).
expression: String,
/// Node's metadata.
metadata: Option<NodeMetadata>,
}
/// Copy the node to the clipboard.
pub fn copy_node(expression: String, metadata: Option<NodeMetadata>) -> FallibleResult {
let text_data = Some(expression.clone());
let content = ClipboardContent::Node(CopiedNode { expression, metadata });
let text_repr = serde_json::to_string(&content)?;
clipboard::write(text_repr.as_bytes(), MIME_TYPE.to_string(), text_data);
Ok(())
}
/// Paste the node from the clipboard at a specific position.
///
/// As pasting is an asynchronous operation, we need to provide a callback for handling possible
/// errors.
pub fn paste_node(graph: &Handle, position: Vector2, on_error: fn(String)) {
clipboard::read(
MIME_TYPE.to_string(),
paste_node_from_custom_format(graph, position, on_error),
plain_text_fallback(graph, position, on_error),
);
}
/// A standard callback for pasting node using our custom format.
fn paste_node_from_custom_format(
graph: &Handle,
position: Vector2,
on_error: impl Fn(String) + 'static,
) -> impl Fn(Vec<u8>) + 'static {
let graph = graph.clone_ref();
let closure = move |content| -> FallibleResult {
let _transaction = graph.module.get_or_open_transaction("Paste node");
let string = String::from_utf8(content)?;
if let Ok(content) = serde_json::from_str(&string) {
match content {
ClipboardContent::Node(node) => {
let expression = node.expression;
let metadata = node.metadata;
graph.new_node_at_position(position, expression, metadata)?;
Ok(())
}
}
} else {
Err(InvalidFormatError.into())
}
};
move |content| {
if let Err(err) = closure(content) {
on_error(format!("Failed to paste node. {err}"));
}
}
}
/// An alternative callback for pasting node from plain text. It is used when [`MIME_TYPE`] is not
/// available in the clipboard, and only if [`PLAIN_TEXT_PASTING_ENABLED`]. Otherwise, it is a
/// noop.
fn plain_text_fallback(
graph: &Handle,
position: Vector2,
on_error: impl Fn(String) + 'static,
) -> impl Fn(String) + 'static {
let graph = graph.clone_ref();
let closure = move |text| -> FallibleResult {
if PLAIN_TEXT_PASTING_ENABLED {
let _transaction = graph.module.get_or_open_transaction("Paste node");
let expression = text;
graph.new_node_at_position(position, expression, None)?;
}
Ok(())
};
move |text| {
if let Err(err) = closure(text) {
on_error(format!("Failed to paste node. {err}"));
}
}
}
// ===============
// === Helpers ===
// ===============
impl Handle {
/// Create a new node at the provided position.
fn new_node_at_position(
&self,
position: Vector2,
expression: String,
metadata: Option<NodeMetadata>,
) -> FallibleResult {
let info = NewNodeInfo {
expression,
doc_comment: None,
metadata,
id: None,
location_hint: double_representation::graph::LocationHint::End,
introduce_pattern: true,
};
let ast_id = self.add_node(info)?;
self.set_node_position(ast_id, position)?;
Ok(())
}
}

View File

@ -58,7 +58,7 @@ impl StatusNotificationPublisher {
pub fn publish_event(&self, label: impl Into<String>) {
let label = label.into();
let notification = StatusNotification::Event { label };
executor::global::spawn(self.publisher.publish(notification));
self.publisher.notify(notification);
}
/// Publish a notification about new process (see [`StatusNotification::ProcessStarted`]).
@ -69,7 +69,7 @@ impl StatusNotificationPublisher {
let label = label.into();
let handle = Uuid::new_v4();
let notification = StatusNotification::BackgroundTaskStarted { label, handle };
executor::global::spawn(self.publisher.publish(notification));
self.publisher.notify(notification);
handle
}
@ -78,7 +78,7 @@ impl StatusNotificationPublisher {
#[profile(Debug)]
pub fn published_background_task_finished(&self, handle: BackgroundTaskHandle) {
let notification = StatusNotification::BackgroundTaskFinished { handle };
executor::global::spawn(self.publisher.publish(notification));
self.publisher.notify(notification);
}
/// The asynchronous stream of published notifications.

View File

@ -20,6 +20,7 @@ use ide_view::graph_editor::component::node as node_view;
use ide_view::graph_editor::component::visualization as visualization_view;
use span_tree::generate::Context as _;
use view::graph_editor::CallWidgetsConfig;
use view::notification::logged as notification;
// ==============
@ -292,6 +293,16 @@ impl Model {
Some((node_id, config))
}
fn node_copied(&self, id: ViewNodeId) {
self.log_action(
|| {
let ast_id = self.state.ast_node_id_of_view(id)?;
Some(self.controller.graph().copy_node(ast_id))
},
"copy node",
)
}
/// Node was removed in view.
fn node_removed(&self, id: ViewNodeId) {
self.log_action(
@ -443,6 +454,14 @@ impl Model {
}
}
fn paste_node(&self, cursor_pos: Vector2) {
fn on_error(msg: String) {
error!("Error when pasting node. {}", msg);
notification::error(msg, &None);
}
self.controller.graph().paste_node(cursor_pos, on_error);
}
/// Look through all graph's nodes in AST and set position where it is missing.
#[profile(Debug)]
fn initialize_nodes_positions(&self, default_gap_between_nodes: f32) {
@ -762,6 +781,7 @@ impl Graph {
// === Changes from the View ===
eval view.node_copied((node_id) model.node_copied(*node_id));
eval view.node_position_set_batched(((node_id, position)) model.node_position_changed(*node_id, *position));
eval view.node_removed((node_id) model.node_removed(*node_id));
eval view.nodes_collapsed(((nodes, _)) model.nodes_collapsed(nodes));
@ -776,8 +796,9 @@ impl Graph {
eval_ view.reopen_file_in_language_server (model.reopen_file_in_ls());
// === Dropping Files ===
// === Dropping Files and Pasting Node ===
eval view.request_paste_node((pos) model.paste_node(*pos));
file_upload_requested <- view.file_dropped.gate(&project_view.drop_files_enabled);
eval file_upload_requested (((file,position)) model.file_dropped(file.clone_ref(),*position));
}

View File

@ -533,7 +533,9 @@ ensogl::define_endpoints_2! {
/// opposed to e.g. when loading a graph from a file).
start_node_creation_from_port(),
// === Copy-Paste ===
copy_selected_node(),
paste_node(),
/// Remove all selected nodes from the graph.
@ -715,6 +717,13 @@ ensogl::define_endpoints_2! {
node_being_edited (Option<NodeId>),
node_editing (bool),
// === Copy-Paste ===
node_copied(NodeId),
// Paste node at position.
request_paste_node(Vector2),
file_dropped (ensogl_drop_manager::File,Vector2<f32>),
connection_made (Connection),
@ -2989,6 +2998,17 @@ fn init_remaining_graph_editor_frp(
}
// === Copy-Paste ===
frp::extend! { network
out.node_copied <+ inputs.copy_selected_node.map(f_!(model.nodes.last_selected())).unwrap();
cursor_pos_at_paste <- cursor.scene_position.sample(&inputs.paste_node).map(|v| v.xy());
out.request_paste_node <+ cursor_pos_at_paste.map(
f!([model](pos) new_node_position::at_mouse_aligned_to_close_nodes(&model, *pos))
);
}
// === Set Node Comment ===
frp::extend! { network

View File

@ -79,6 +79,9 @@ pub const SHORTCUTS: &[(ensogl::application::shortcut::ActionType, &str, &str, &
(Release, "!read_only", "cmd", "edit_mode_off"),
(Press, "!read_only", "cmd left-mouse-button", "edit_mode_on"),
(Release, "!read_only", "cmd left-mouse-button", "edit_mode_off"),
// === Copy-paste ===
(Press, "!node_editing", "cmd c", "copy_selected_node"),
(Press, "!read_only", "cmd v", "paste_node"),
// === Debug ===
(Press, "debug_mode", "ctrl d", "debug_set_test_visualization_data_for_selected_node"),
(Press, "debug_mode", "ctrl n", "add_node_at_cursor"),

View File

@ -72,35 +72,97 @@ export function writeText(text) {
}
}
/// Write custom `data` payload to the clipboard. Data will be saved as a `Blob` with `mimeType`.
/// If `textData` is not empty, an additional clipboard item will be written with the `text/plain` type.
///
/// Unlike `writeText`, there are no special fallbacks in case of errors or the clipboard being unavailable.
/// If writing did not succeeed, the function will simply log an error to the console.
export function writeCustom(mimeType, data, textData) {
if (!navigator.clipboard) {
console.error('Clipboard API not available.')
} else {
const blob = new Blob([data], { type: mimeType })
const payload = { [blob.type]: blob }
if (typeof textData === 'string' && textData !== '') {
payload['text/plain'] = new Blob([textData], { type: 'text/plain' })
}
navigator.clipboard.write([new ClipboardItem(payload)]).then(
() => {},
err => {
console.error('Could not write to clipboard.', err)
}
)
}
}
/// Firefox only supports reading the clipboard in browser extensions, so it will
/// only work with `cmd + v` shortcut. To learn more, see the
/// [MSDN compatibility note](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/readText).
let lastPaste = ''
let lastTextPaste = ''
function init_firefox_fallback() {
// Checking whether the window is defined. It could not be defined if the program is run in
// node, for example to extract the shaders.
if (typeof window !== 'undefined') {
window.addEventListener('paste', event => {
lastPaste = (event.clipboardData || window.clipboardData).getData('text')
lastTextPaste = (event.clipboardData || window.clipboardData).getData('text')
})
}
}
export function readText(callback) {
if (!navigator.clipboard) {
callback(lastPaste)
callback(lastTextPaste)
} else {
navigator.clipboard.readText().then(
function (text) {
callback(text)
},
function (err) {
callback(lastPaste)
callback(lastTextPaste)
}
)
}
}
/// Read a custom payload of `expectedMimeType` from the clipboard, passing it to `whenExpected` callback.
/// If there is no value of `expectedMimeType` in the payload, use `plainTextFallback` callback instead.
///
/// Unlike `readText`, there are no special fallbacks in case of errors or the clipboard being unavailable.
/// If reading did not succeeed, the function will simply log an error to the console.
export function readCustom(expectedMimeType, whenExpected, plainTextFallback) {
if (!navigator.clipboard) {
console.error('Clipboard API not available.')
} else {
readCustomImpl(expectedMimeType, whenExpected, plainTextFallback)
}
}
/// Helper function for `readCustom`, see its documentation.
async function readCustomImpl(expectedMimeType, whenExpected, plainTextFallback) {
try {
const data = await navigator.clipboard.read()
for (const item of data) {
if (item.types.includes(expectedMimeType)) {
const blob = await item.getType(expectedMimeType)
const buffer = await blob.arrayBuffer()
whenExpected(new Uint8Array(buffer))
return
}
}
// We use a separate loop to make sure `expectedMimeType` has a priority, no matter the order of items.
for (const item of data) {
if (item.types.includes('text/plain')) {
const blob = await item.getType('text/plain')
const text = await blob.text()
plainTextFallback(text)
return
}
}
} catch (error) {
console.error('Error while reading clipboard.', error)
}
}
// ======================
// === Initialization ===
// ======================

View File

@ -1,7 +1,25 @@
//! Clipboard management utilities.
//!
//! Please note:
//! - Every function here uses the [Clipboard API](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard_API)
//! under the hood.
//! - Every function is asynchronous. The delay for receiving or sending data may be caused for
//! example by waiting for permission from the user.
//! - Every function will probably display a permission prompt to the user for the first time it is
//! used.
//! - The website has to be served over HTTPS for these functions to work correctly.
//! - These functions needs to be called from within user-initiated event callbacks, like mouse or
//! key press. Otherwise it may not work.
//! - Web browsers do not support MIME types other than `text/plain`, `text/html`, and `image/png`
//! in general. However, using
//! [Clipboard pickling](https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md),
//! we can practically use any MIME type.
//!
//! To learn more, see this [StackOverflow question](https://stackoverflow.com/questions/400212/how-do-i-copy-to-the-clipboard-in-javascript).
use crate::prelude::*;
use js_sys::Uint8Array;
use wasm_bindgen::prelude::wasm_bindgen;
use wasm_bindgen::prelude::Closure;
@ -11,7 +29,13 @@ use wasm_bindgen::prelude::Closure;
// === Types ===
// =============
/// MIME type of the data.
pub type MimeType = String;
/// The data to be written to the clipboard.
pub type BinaryData<'a> = &'a [u8];
type ReadTextClosure = Closure<dyn Fn(String)>;
type ReadClosure = Closure<dyn Fn(Vec<u8>)>;
@ -26,53 +50,107 @@ extern "C" {
#[allow(unsafe_code)]
fn readText(closure: &ReadTextClosure);
#[allow(unsafe_code)]
fn writeCustom(mime_type: String, data: Uint8Array, text_data: String);
#[allow(unsafe_code)]
fn readCustom(
expected_mime_type: String,
when_expected: &ReadClosure,
plain_text_fallback: &ReadTextClosure,
);
}
/// Write the provided text to the clipboard. Please note that:
/// - It uses the [Clipboard API](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard_API)
/// under the hood.
/// - This is an asynchronous function. The results will not appear in the clipboard immediately.
/// The delay may be caused for example by waiting for permission from the user.
/// - This will probably display a permission prompt to the user for the first time it is used.
/// - The website has to be served over HTTPS for this function to work correctly.
/// - This function needs to be called from within user-initiated event callbacks, like mouse or key
/// press. Otherwise it will not work.
/// Write the provided data to the clipboard, using the provided MIME type.
/// If `text_data` is present, it will be added to the clipboard with a `text/plain` MIME type.
///
/// Moreover, in case something fails, this function implements a fallback mechanism which tries
/// See the module documentation for mode details.
///
/// - Unlike `write_text`, there is no special fallback mechanism in case of failures or unavailable
/// clipboard. The function will simply report an error to the console.
pub fn write(data: BinaryData<'_>, mime_type: MimeType, text_data: Option<String>) {
let data = Uint8Array::from(data);
writeCustom(mime_type, data, text_data.unwrap_or_default());
}
/// Read the arbitrary binary data from the console. It is expected to have `expected_mime_type`.
/// If the value of such type is not present in the clipboard content, the `plain/text` MIME type
/// is requested and the result is passed to the `plain_text_fallback` callback.
///
/// See the module documentation for more details.
///
/// - Unlike `read_text`, there is no special fallback mechanism in case of failures or unavailable
/// clipboard. The function will simply report an error to the console.
pub fn read(
expected_mime_type: MimeType,
when_expected: impl Fn(Vec<u8>) + 'static,
plain_text_fallback: impl Fn(String) + 'static,
) {
let when_expected_handler = create_handler_binary(when_expected);
let fallback_handler = create_handler_string(plain_text_fallback);
readCustom(
expected_mime_type,
when_expected_handler.borrow().as_ref().unwrap(),
fallback_handler.borrow().as_ref().unwrap(),
);
}
/// Write the provided text to the clipboard.
///
/// See the module documentation for more details.
///
/// In case something fails, this function implements a fallback mechanism which tries
/// to create a hidden text field, fill it with the text and use the obsolete
/// [Document.execCommand](https://developer.mozilla.org/en-US/docs/Web/API/Document/execCommand)
/// function.
///
/// To learn more, see this [StackOverflow question](https://stackoverflow.com/questions/400212/how-do-i-copy-to-the-clipboard-in-javascript).
pub fn write_text(text: impl Into<String>) {
let text = text.into();
writeText(text)
}
/// Read the text from the clipboard. Please note that:
/// - It uses the [Clipboard API](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard_API)
/// under the hood.
/// - This is an asynchronous function. The callback with the text will be called when the text will
/// be ready. The delay may be caused for example by waiting for permissions from the user.
/// - This will probably display a permission prompt to the user for the first time it is used.
/// - The website has to be served over HTTPS for this function to work correctly.
/// - This function needs to be called from within user-initiated event callbacks, like mouse or key
/// press. Otherwise it will not work.
/// Read the text from the clipboard.
///
/// Moreover, this function works in a very strange way in Firefox.
/// See the module documentation for more details.
///
/// This function works in a very strange way in Firefox.
/// [Firefox only supports reading the clipboard in browser extensions](https://developer.mozilla.org/en-US/docs/Web/API/Clipboard/readText).
/// In such case this function fallbacks to the `paste` event. Whenever it is triggered, it
/// remembers its value and passes it to the callback. This means, that in Firefox this function
/// will work correctly only when called as a direct action to the `cmd + v` shortcut.
///
/// To learn more, see this [StackOverflow question](https://stackoverflow.com/questions/400212/how-do-i-copy-to-the-clipboard-in-javascript).
pub fn read_text(callback: impl Fn(String) + 'static) {
let handler = create_handler_string(callback);
readText(handler.borrow().as_ref().unwrap());
}
// ===============
// === Helpers ===
// ===============
fn create_handler_string(
callback: impl Fn(String) + 'static,
) -> Rc<RefCell<Option<Closure<dyn Fn(String)>>>> {
let handler: Rc<RefCell<Option<ReadTextClosure>>> = default();
let handler_clone = handler.clone_ref();
let closure: Closure<dyn Fn(String)> = Closure::new(move |result| {
let closure: ReadTextClosure = Closure::new(move |result| {
*handler_clone.borrow_mut() = None;
callback(result);
});
*handler.borrow_mut() = Some(closure);
readText(handler.borrow().as_ref().unwrap());
handler
}
fn create_handler_binary(
callback: impl Fn(Vec<u8>) + 'static,
) -> Rc<RefCell<Option<Closure<dyn Fn(Vec<u8>)>>>> {
let handler: Rc<RefCell<Option<ReadClosure>>> = default();
let handler_clone = handler.clone_ref();
let closure: ReadClosure = Closure::new(move |result| {
*handler_clone.borrow_mut() = None;
callback(result);
});
*handler.borrow_mut() = Some(closure);
handler
}